JITOMA lets robots build scene maps on-demand, slashing latency by 60%

Conventional 3D Scene Graph (3DSG) pipelines for embodied agents build everything in advance, then filter—a process that kills real-time performance on edge devices due to perceptual saturation (observing too much redundant data). To fix this, a team of researchers (Yue Chang, Rufeng Chen, et al.) present JITOMA (Just-In-Time On-demand Memory Activation). The framework unifies task reasoning, perception, and memory into a just-in-time growth process. At the front-end, a top-down task heatmap filters continuous observations, routing only minimal streams to maintain a global foundation of low-cost, dormant anchors. When a cognitive query hits, a backend Large Language Model parses the robotic intent to dynamically awaken task-relevant anchors, triggering resource-intensive operations like dense node captioning and functional inference exclusively within the activated local subgraph.

To rigorously test dynamic capabilities and study perceptual saturation trade-offs, the authors introduce JITOMA-Bench, a comprehensive suite for long-horizon multi-tasking and complex multi-step reasoning. Extensive experiments show that JITOMA substantially reduces active graph size and captioning latency, while keeping processing time stable even under long-horizon task switching. This represents a paradigm shift from "build everything then filter" to "grow only what you need," making real-time robotic perception feasible on edge hardware without sacrificing accuracy.