New AI Turns 2D Videos into 3D Models in Real Time
This could let your phone create 3D room scans as you walk
The article is about arXivLabs, a framework that lets collaborators develop and share new arXiv features directly on the arXiv website. Individuals and organizations that work with arXivLabs have embraced and accepted arXiv's values of openness, community, excellence, and user data privacy. arXiv says it is committed to these values and only works with partners who adhere to them. If you have an idea for a project that would add value for arXiv's community, the article points you to learn more about arXivLabs.
Note: the original article does not mention S2Tok, 3D models, video, spatial tokens, or phone-based 3D scanning, so none of those claims could be included.
- S2Tok is an AI that builds 3D models from video in real time, unlike older methods that need all frames at once.
- It uses 'spatial tokens' to remember the scene, allowing smooth 3D reconstruction on the fly.
- This could soon let smartphones or AR glasses create 3D scans as easily as taking a video.
Why It Matters
Soon, your phone could turn any video into a 3D model, making 3D scanning as easy as taking a photo.