Auto-JEPA: Latent world model boosts self-driving planning with 91.3 PDMS

Auto-JEPA, introduced by Jiwei Yang and colleagues in a new arXiv paper, challenges the conventional approach to autonomous-driving world models. Existing systems perform dense prediction—reconstructing entire future videos, occupancy grids, or BEV representations. Auto-JEPA argues that planning doesn't need the full future world, only the elements that influence the ego vehicle's next actions. It achieves this with a joint-embedding predictive architecture: given visual observations, egomotion history, and navigation commands, it predicts an intent embedding aligned with the latent representation of the future ego trajectory. The predicted intent then retrieves executable trajectories from a fixed memory bank, ranked by a scene-conditioned selection module.

The design is deliberately lean. Auto-JEPA keeps the visual encoder frozen, requires no explicit perception annotations, and eliminates the learned trajectory generator. Only task-specific modules for trajectory representation, intent prediction, and candidate selection are optimized. That simplicity yields strong results: 91.3 PDMS on NAVSIM v1 and 89.1 EPDMS on NAVSIM v2. Semantic occlusion experiments reveal the model's planning focus—masking dynamic-agent regions causes an intent change 2.97x larger than equal-area random masking. Occluding vehicles that actually affect future driving shifts both the predicted intent and the chosen trajectory, while non-influential vehicles leave them essentially unchanged. This confirms that future-intent prediction naturally prioritizes planning-relevant visual features, enabling high-quality planning without costly dense future-world reconstruction.