VideoCoCo uses Blender code to improve physical consistency in video generation

Text-to-video models produce stunning visuals but often violate physical laws because they infer scene dynamics implicitly from compressed text prompts. VideoCoCo, developed by a team of 28 researchers, tackles this by using executable Blender code as a process-level chain of thought. The framework's dual-engine architecture separates reasoning from realization: a coding agent writes a Blender program that explicitly defines the scene and its temporal evolution, then an executable simulation engine runs that program to generate a deterministic spatiotemporal draft. A separate generative video engine transforms the draft into a photorealistic video using draft-conditioned editing. This decomposition makes the intermediate representation fully inspectable and controllable, addressing the sparsity and non-executability of prior chain-of-thought approaches.

To support the pipeline, the team created VideoCoCo-3K, a curated dataset of draft-instruction-target triplets that adapts video editors to simulated inputs. The empirical results are significant: VideoCoCo improves the OmniWeaving baseline from 0.475 to 0.558 on PhyGenBench and from 52.18 to 77.88 on VBench-2.0, achieving the best average score on both benchmarks. This demonstrates that executable code offers a more effective intermediate representation for physically consistent video generation than text plans or sparse visual states. For professionals, the practical takeaway is clear—AI video generation is moving beyond black-box prompting. With VideoCoCo, users can inspect, debug, and modify the underlying Blender code that drives physics, enabling precise control over object interactions, camera motion, and scene dynamics. The paper is available on arXiv with code, data, and demo links.