SeeSE3: Vision models inherently encode 3D space without explicit training
Self-supervised models already embed 3D Euclidean geometry in their latent spaces.
The paper SeeSE3 investigates whether vision foundation models construct representations that reflect the intrinsic properties of 3D Euclidean space. It proposes probes including a mutual neighborhood metric and a Poincaré Adapter to evaluate this relation from topological and geometric perspectives. The authors show that self-supervised vision models possess latent subspaces strongly correlated with three-dimensional Euclidean space when probed correctly. Building on this insight, they introduce Latent-Space Navigation techniques that perform visual odometry and localization purely in the latent space, bypassing the need for explicit 3D reconstruction.
- Mutual neighborhood metric aligns feature-space neighborhoods with 3D spatial topology across self-supervised models
- Poincaré Adapter demonstrates linear decodability of camera motion (SE(3) transformations) from latent displacements
- Latent-Space Navigation performs visual odometry and localization without explicit 3D reconstruction, achieving competitive accuracy
Why It Matters
Enables lighter, faster spatial AI for robotics, AR, and autonomous navigation by skipping explicit 3D reconstruction.