Closing the Loop: Training-Free Consistency for 3D Video Generation

Autoregressive video generation models have shown promise for transforming 3D engine renderings—like depth maps and untextured geometry—into photorealistic videos for gaming and immersive content. However, these models often suffer from 'revisit inconsistency': when the camera returns to a previously visited location after its bounded KV cache has evicted that context, the model regenerates inconsistent appearance, even though the conditioning renderings (depth, pose) remain perfectly aligned with the 3D world. Current solutions typically require costly post-training or fine-tuning.

In a new paper, researchers Wenchao Ma, Changran Liu, Sharon X. Huang, and Haomiao Jiang introduce a training-free method that exploits correspondences already provided by the 3D engine. Temporal correspondence retrieves pose-matched historical latent chunks into the KV cache as loop-closure memory, while spatial correspondence from camera pose and depth reprojection biases token-level attention toward geometrically corresponding regions. Demonstrated on loop-closure trajectories from TartanAir and TartanGround datasets, the method outperforms existing training-free baselines on revisit consistency without degrading overall video quality. This approach makes long-horizon, consistent 3D video generation practical for real-world applications.