μ0 world model teaches robots via 3D traces, no action labels needed

World models that predict how actions change the physical world typically rely on either dense pixel prediction (costly and appearance-heavy) or direct action modeling (needs embodiment-specific labels, limiting scalability). μ0 sidesteps both by forecasting smooth 3D trajectories for salient interaction points—objects, tools, hands, and contact regions. This compact, embodiment-agnostic representation allows robots to learn motion dynamics without ever seeing action labels during pretraining.

The key enabler is TraceExtract, a system that automatically generates 3D supervision from unlabeled video. It selects keypoints in each frame, constructs globally aligned 3D traces, and associates motion segments with hierarchical language captions. μ0 then uses a pretrained vision-language backbone paired with a modular trace expert, representing each query via B-spline control points to predict future traces. This pipeline lets μ0 train on massive, diverse video sources without manual annotation.

In experiments, μ0 outperformed baselines in both 2D and 3D trace prediction, including specialized trace models and tokenized vision-language methods. More importantly, because μ0 is frozen and reusable after pretraining, it can be combined with lightweight action experts for different robot embodiments. Despite never seeing action labels, the resulting trace-conditioned policies rival the performance of large VLA models like π0, which require explicit action supervision. The results establish 3D traces as a scalable, transferable representation for cross-embodiment robot manipulation.