VoxelHound: Quadruped robots see 360° with +4.16 mIoU occupancy prediction

A team of researchers from multiple institutions has introduced VoxelHound, a panoramic multimodal semantic occupancy prediction framework specifically designed for quadruped robots. Alongside this, they present PanoMMOcc, billed as the first real-world panoramic multimodal occupancy dataset for legged locomotion, capturing four distinct sensing modalities across diverse scenes. The work, posted on arXiv, addresses a major gap: existing occupancy prediction methods are optimized for wheeled autonomous driving and rely heavily on RGB cues, making them fragile in dynamic, unstructured environments where quadruped robots often operate.

VoxelHound is built on two key innovations. The Vertical Jitter Compensation (VJC) module mitigates severe viewpoint perturbations caused by the robot's body pitch and roll during walking, ensuring consistent spatial reasoning. The Multimodal Information Prompt Fusion (MIPF) module then effectively integrates panoramic visual inputs with auxiliary modalities, such as depth or lidar, to boost volumetric occupancy prediction. In experiments on the PanoMMOcc benchmark, VoxelHound achieves state-of-the-art performance, outperforming prior methods by a solid +4.16 mIoU.

The authors have committed to releasing both the dataset and code publicly, which should accelerate research in embodied 3D perception for legged robots. This moves quadruped robots closer to reliable navigation and interaction in complex, real-world environments, from industrial inspection to search-and-rescue.