13 Researchers Just Unified the Fragmented World of Spatial AI — Here's What They Found

3D Scene Graphs (3DSGs) have emerged as a powerful representation for spatial AI, bridging geometric grounding with semantic and relational abstractions of environments. However, the field has been fragmented, with different communities adopting distinct formulations, pipelines, and evaluation metrics. A new comprehensive survey by Dennis Rotondi and 12 co-authors from institutions including MIT, TU Munich, and Meta aims to unify the field. The paper formalizes 3DSGs under a common definition, analyzing node/edge attributes, hierarchical structure, dynamic scenes, and affordance-aware extensions. It also reviews how 3DSGs are built from raw sensory data (e.g., RGB-D, LiDAR) and examines downstream applications like manipulation, navigation, and task planning.

The survey goes beyond simple taxonomy by critically evaluating intrinsic graph quality and task-level performance metrics. The authors identify open challenges that hinder robust real-world deployment: scalability, dynamic environments, standardization of benchmarks, and integration with large language models. They provide a dedicated website to organize and extend the content. This work is particularly timely as autonomous systems demand richer spatial understanding. By offering a unified language and highlighting future directions, the survey serves as a roadmap for researchers working on perception, planning, and scene understanding. It is published as an invited article in the Annual Review of Control, Robotics, and Autonomous Systems.