Bootstrap Perception Under Hardware Depth Failure for Indoor Robot Navigation

A team of researchers has unveiled a novel AI architecture that allows robots to 'bootstrap' their own perception when critical hardware fails. The system, detailed in a new arXiv paper, addresses a pervasive problem in indoor robotics: time-of-flight depth cameras often fail on reflective surfaces like glass or polished floors, losing up to 78% of their depth pixels. A 2D LiDAR alone can't see obstacles above its scan plane, creating dangerous blind spots. The researchers' key insight was to exploit a self-referential property of the failure—the sensor's own surviving valid pixels are used to calibrate a learned, monocular depth model to metric scale. This allows the system to fill its own gaps without needing external calibration data.

The architecture creates a 'failure-aware sensing hierarchy.' It remains conservative, using LiDAR as the primary geometric anchor and hardware depth where it's valid. The AI-generated, learned depth enters the scene only where sensors have definitively failed. This selective fusion proved highly effective. In evaluations in corridors and with dynamic pedestrians, it increased costmap obstacle coverage by 55-110% compared to using LiDAR alone. Crucially, the team distilled the system into a compact student model that runs at a blazing 218 frames per second on an NVIDIA Jetson Orin Nano, a common embedded AI computer. In closed-loop simulation, it matched the performance of a ground-truth depth baseline, achieving 9 out of 10 navigation successes with zero collisions, but at a fraction of the computational cost of a large foundation model.