New AI Turns Any Single Photo Into an Accurate 3D Model

Researchers built a system called OmniPoint that recovers metric 3D geometry from a single monocular image. What makes it different: it's a unified framework designed to generalize metric reconstruction across diverse imaging sensors — including pinhole, fisheye, and equirectangular projections — while accommodating varying geometric priors. To get there, it abandons conventional planar depth regression in favor of a decoupled ray and distance representation and a decoupled training objective that explicitly separates the camera projection model from the scene structure. The team also introduces a bidirectional augmentation strategy to bridge labeled perspective data and unlabeled omnidirectional domains in 3D space, plus a mechanism using learnable input state embeddings and vectorized Gaussian smoothing to inject optional inputs like camera intrinsics or sparse depth without destabilizing the network. According to the article, extensive experiments show OmniPoint achieves state-of-the-art zero-shot performance across multiple benchmarks. The paper is by Botao Ye, Marc Pollefeys, Ming-Hsuan Yang, and Abhijit Kundu, and is listed as an ECCV 2026 submission.