AI Robot Brains Just Got a High-Stakes Test
If robots are going to do your chores, we need to know if they really understand what they're seeing.
A new 3D-grounded benchmark for embodied world models reveals that even the best video-predicting AI still falls short of real-world grounding. The benchmark, built on 50 manipulation tasks with 25,000 multi-view ground-truth videos, measures generated rollouts across pixel fidelity, 3D geometry, state understanding, and task completeness. Among four representative models, Cosmos 3 scored highest with a RoboPhyscore of 0.6330—92.7% of the ground-truth score—while state- and execution-grounded metrics exposed major failures that perceptual and vision-language-model judgments miss. The score also strongly matched human evaluation, highlighting the need for grounded, execution-aware assessment.
- Researchers created a new 3D test for AI robots to see if they truly understand the physical world, not just what things look like in video
- Only one AI, Cosmos 3, scored high enough (92.7%) to be trusted with real-world tasks like cooking or handling fragile objects
- Most AI robots still fail in subtle ways—like thinking a heavy box is light—making them unreliable for dangerous or delicate jobs
Why It Matters
This test decides whether AI robots can safely handle real chores, healthcare, or work—before they’re let loose in your home or office.