AI models fake spatial understanding via instruction leakage, study reveals

A new preprint from Wang, Wei, and Ling (arXiv:2607.06925) reveals a critical flaw in how AI world models learn spatial relations. The researchers trained a compact world model to condition on language goals like 'put the red block left of the blue block.' The model appeared to achieve 0.90 relation-readout accuracy, suggesting it understood the spatial arrangement. However, when the goal instruction was withheld, accuracy plummeted to 0.27 across three seeds. A counterfactual instruction made the model predict anchors following the false instruction 94.5% of the time, while the true scene accounted for only 2.3%. The phenomenon, termed 'instruction leakage,' means the model is simply transcribing the instruction rather than grounding the relation in the visual input.

The researchers tested this across tabletop environments, the BabyAI benchmark, and a Language-Table forward-dynamics model. They found leakage occurs whenever the scored quantity can be read directly from the instruction, independent of how predictive the non-instruction inputs are. Their proposed fix is a 'goal-free dynamics' approach: keep the language goal out of the dynamics model (it belongs in the planner's cost function) and instead supervise the read path separately. This restores genuine grounding — achieving 0.88 accuracy both with and without the goal instruction. The detection protocol and remedy generalize to any goal-conditioned world model where the instruction names the scored quantity, offering a practical safeguard for embodied AI and robotics systems.