Sparse3DTrack: Monocular 3D Object Tracking Using Sparse Supervision

A research team has introduced Sparse3DTrack, the first framework for monocular 3D object tracking that operates with sparse supervision. Monocular 3D tracking is crucial for autonomous systems to understand scene dynamics from video, but existing state-of-the-art methods rely on dense, expensive 3D annotations across long sequences. Sparse3DTrack fundamentally addresses this scalability bottleneck by requiring only a handful of labeled frames—at most four ground truth annotations per object track—instead of frame-by-frame labeling.

The framework cleverly decomposes the complex task into two sequential, learnable sub-problems: 2D query matching and 3D geometry estimation. Both components exploit the inherent spatio-temporal consistency of video sequences to amplify a sparse set of human-labeled samples. By learning rich 2D and 3D representations, the model can automatically generate high-quality 3D pseudolabels across entire videos, effectively transforming a few annotations into a dense, usable training signal.

This generated pseudolabel data enables existing, fully-supervised 3D trackers to perform effectively under extreme label scarcity. In extensive experiments on major autonomous driving datasets, the method demonstrated significant improvements, boosting tracking performance by up to 15.50 percentage points (p.p.) on standard benchmarks. The approach represents a major step toward practical and scalable perception for robotics and autonomous vehicles, where collecting dense 3D ground truth is prohibitively costly.