NavVerse: new AI benchmark reveals robots fail at indoor-to-outdoor navigation

NavVerse is a new benchmark from researchers at the University of Michigan (Wu et al.) designed to test robots on continuous navigation from indoor to outdoor environments—a critical capability for delivery, campus patrol, and emergency response. The benchmark includes 100 indoor scenes, 50 urban outdoor scenes, and 50 indoor-to-outdoor scenes, totalling 10,000 episodes across three tasks: Object Navigation, Vision-and-Language Navigation (VLN), and Place Navigation (PlaceNav). Agents must search for semantic points of interest such as restaurants or banks while being evaluated on task success, path efficiency, and safety through executable robot interfaces.

Zero-shot experiments using reinforcement learning (RL), vision-language-action models (VLAs), and modular methods reveal that current agents are far from solving cross-context navigation. End-to-end VLAs achieved the highest zero-shot success rates, but modular approaches provided the strongest safety profile—highlighting a trade-off between performance and risk. Critically, PlaceNav results showed a clear drop in success when transitioning from outdoor-only scenes to indoor-to-outdoor scenes, confirming that adaptation across environmental contexts remains a major bottleneck. This benchmark exposes a gap that must be addressed before robots can reliably operate in real-world mixed environments.