MindTopo benchmark reveals VLMs ace static topology but fail planning

MindTopo is a new benchmark designed to evaluate whether multimodal large language models possess topological intuition—the structural understanding of connectivity, enclosure, order, separation, and knots. Unlike standard spatial benchmarks that test Euclidean properties like distance and direction, MindTopo draws on cognitive science (Piaget's classification) to probe how models handle relationships that persist under deformation. Each of its five categories is tested at two cognitive levels: reasoning tasks ask models to analyze static rendered scenes, while planning tasks require them to manipulate simulated environments through legal actions—such as rotating pipe segments, drawing separation paths, or untangling ropes without passing strands through one another.

Across a broad set of proprietary and open-weight models, results reveal a substantial gap. Models can sometimes recognize connected paths, enclosed regions, or true knots in a single image, but that understanding breaks down during planning: they lose track of structural relationships as scenes change or propose actions that violate physical constraints. Because all scenes are generated by controlled simulators with exact ground truth and adjustable difficulty, MindTopo cleanly separates failures caused by visual complexity from those caused by a model's inability to maintain relationships over time. This suggests the bottleneck is not perception but sustained relational reasoning, a critical warning for applications in robotics and interactive environments where knowing what stays connected, enclosed, ordered, or knotted is essential.