Predictive Spatio-Temporal Scene Graphs for Semi-Static Scenes

A team from the University of Montreal (Saavedra-Ruiz et al.) has developed PredictiveGraphs, a framework that equips robots with tempo-spatio-semantic reasoning over semi-static scenes. The key innovation is Perpetua*, a Bayesian filter integrated into a 3D scene graph where nodes represent objects and edges encode spatio-semantic relationships with probabilistic transitions. This allows the system to learn cyclic behaviors—like a mug moving from cupboard to countertop to sink and back daily—and predict future object states. Unlike most spatio-semantic representations that lack temporal reasoning, PredictiveGraphs can anticipate changes even in the presence of distributional shifts (e.g., a different placement pattern).

The method was validated in both simulation and a real-world dynamic navigation task spanning three weeks with observations taken every two hours. In both settings, PredictiveGraphs significantly outperformed baseline approaches at predicting future environment states. This capability is crucial for robots operating in homes or offices where objects are frequently moved but in structured patterns (e.g., dishes, tools). By forecasting where items will be, robots can plan navigation and manipulation tasks more efficiently. The paper is available on arXiv (2605.00121) and represents a practical step toward long-horizon autonomy in semi-structured environments.