ICAT: Incident-Case-Grounded Adaptive Testing for Physical-Risk Prediction in Embodied World Models

A research team from multiple institutions has published a paper introducing ICAT (Incident-Case-Grounded Adaptive Testing), a novel framework designed to rigorously evaluate the physical-risk prediction capabilities of embodied AI world models. These video-generative models, which simulate environments for robot planning and policy training, are increasingly used as neural simulators but have shown dangerous shortcomings. The researchers found these models consistently downplay or completely omit critical danger cues and severe outcomes when imagining hazardous scenarios, which could lead to unsafe preferences during AI training.

ICAT addresses this by building structured "risk memories" from real-world incident reports and safety manuals, then using retrieval and composition techniques to generate constrained test cases with causal chains and severity labels. When tested against this benchmark, mainstream world models failed to reliably predict physical risks, frequently missing key mechanisms and triggering conditions while miscalibrating severity assessments. The gap between current model performance and the reliability required for safety-critical embodied deployment—like autonomous vehicles or physical robots—remains significant, highlighting a major barrier to real-world AI integration.