AV1 motion vectors boost 3D Gaussian Splatting with 8x denser point clouds

3D Gaussian Splatting (3DGS) has become a leading framework for real-time, photorealistic scene reconstruction, but its quality heavily depends on the initial point cloud from Structure-from-Motion (SfM). Traditional SfM pipelines like COLMAP are computationally expensive and produce sparse points in textureless regions, limiting 3DGS accuracy and convergence speed. A new paper from Trinity College Dublin proposes a clever alternative: use motion vectors already computed by the AV1 video codec to drive dense feature matching, bypassing the need for expensive exhaustive matching while maintaining geometric robustness.

Their pipeline generates point clouds up to eight times denser than classical SfM, which directly translates to better 3DGS reconstructions. In tests, the method achieved a 9-point increase in VMAF (a video quality metric) and a 63% reduction in training time required to reach baseline quality. This approach makes high-fidelity 3D scene reconstruction more accessible by reducing both computational overhead and reliance on costly SfM preprocessing.