RoboGaze: Multi-agent VLM framework boosts robot video evaluation by 43 points

Evaluating synthetic videos from robot world models is notoriously difficult—visually realistic outputs often violate physics, temporal logic, or task constraints. Existing metrics and monolithic VLM judges fail to generalize or provide actionable diagnostics. To address this, a team of researchers led by Minh-Loi Nguyen developed RoboGaze, a novel training-free framework that orchestrates multiple VLM agents to deliver structured, interpretable evaluations. The system operates in three stages: first, it grounds the task instruction in the scene; then, it routes the video to dimension-specific specialist agents; finally, a critic verifier validates the outputs. The result is a temporally localized glitch report organized under a newly created 6-dimension, 30-type robotics-specific taxonomy.

RoboGaze was benchmarked on a human-validated dataset of 382 clips spanning simulated and real-world multi-view manipulation tasks. Across eight open-source and proprietary VLM backbones, the framework dramatically outperformed zero-shot baselines, improving description-F1 by up to +43 points and temporal alignment (F1×IoU) by up to +37 points—closing approximately 85% of the gap to the human ceiling. Notably, the critic verifier solved the 'cry-wolf' false-positive problem common in standard VLMs, raising clean-clip accuracy from under 25% to over 80%. RoboGaze thus provides a scalable, highly interpretable diagnostic tool for rigorously evaluating robot world models.