Sub-JEPA boosts LeCun's LeWorldModel by up to 10.7% with a subspace fix
A simple regularization tweak that adds no new hyperparameters yet outperforms across all benchmarks.
World models are crucial for planning in AI, as they learn compact latent representations without full pixel reconstruction. LeCun's group at NYU previously developed LeWorldModel (LeWM), which uses end-to-end JEPA training with an isotropic Gaussian prior over the entire latent space to prevent collapse. However, this global high-dimensional prior is mismatched to real environment dynamics, which live on low-dimensional manifolds. The flaw becomes especially evident on tasks with low intrinsic dimension, such as the Two-Room benchmark, where LeWM struggles.
The new method, Sub-JEPA, addresses this by applying the same Gaussian regularization inside multiple frozen random orthogonal subspaces instead of across the full latent space. This relaxes the global constraint while preserving the anti-collapse benefit. Remarkably, no new hyperparameters are added—the objective remains the same two-term loss. Sub-JEPA consistently outperforms LeWM across all four evaluated benchmarks, with the largest gain of +10.7 percentage points on Two-Room. Emergent improvements include straighter latent trajectories and better decodability of physical states, suggesting that the fix yields more structured and interpretable representations.
- Sub-JEPA applies Gaussian regularization inside frozen random orthogonal subspaces, not globally, fixing LeWM's prior mismatch.
- Achieves up to +10.7 percentage points improvement on the Two-Room benchmark, with consistent gains across all four tasks.
- Emergent benefits include straighter latent trajectories and improved physical state decodability, with no new hyperparameters.
Why It Matters
A simple architectural tweak that makes world models more efficient for planning in real-world, low-dimensional environments.