3DSPA: A 3D Semantic Point Autoencoder for Evaluating Video Realism

Researchers Bhavik Chandna and Kelsey R. Allen have introduced 3DSPA, a novel 3D Semantic Point Autoencoder designed to automate the evaluation of realism in AI-generated videos. As video generation models from companies like OpenAI (Sora), Runway, and Pika Labs advance rapidly, the need for robust, automated evaluation metrics has become critical. Current methods rely heavily on manual human annotation or limited bespoke datasets, creating a bottleneck for development. 3DSPA addresses this by providing a framework that captures both scene semantics and coherent 3D structure without requiring a reference video, enabling scalable assessment for applications from robotics to film-making.

The 3DSPA model works by integrating 3D point trajectories, depth information, and semantic features from models like DINO into a unified spatiotemporal representation. This allows it to model object motion and scene dynamics to assess physical plausibility and temporal consistency. Experiments demonstrate that 3DSPA reliably identifies videos that violate physical laws, is more sensitive to motion artifacts than previous techniques, and shows stronger alignment with human judgments of quality across multiple datasets. The release of its code and pretrained weights will provide the AI community with a powerful new benchmark for developing and comparing generative video models, moving evaluation beyond simple pixel-level metrics toward understanding physical and semantic coherence.