ICML paper formalizes binding problem in Vision Transformers

Vision Transformers (ViTs) power many modern AI vision systems, but they have a known weakness: they often misattribute features to the wrong object—like thinking a blue square is part of a red circle. This 'binding problem'—how to correctly associate features (color, shape) with their respective objects—has been elusive to define formally. In a new paper accepted to ICML 2026, researchers led by Lianghuan Huang introduce an information-theoretic framework that quantifies binding information in model representations. They define binding as the mutual information between feature representations and the assignment of features to objects, providing a precise mathematical tool to measure how well a model 'binds' features together. This work bridges a gap between neuroscience and computer vision, where the binding problem has long been studied in human perception.

The team also developed a probing method to extract binding information from different components of ViTs, such as the [CLS] token and spatial tokens. They tested several pre-trained ViT architectures on datasets designed to challenge binding: scenes with feature sharing (multiple objects with the same color), occlusion, and natural images. Their results show that while ViTs do capture some binding information, they fail significantly when objects share features—a common real-world scenario. The study demonstrates that binding is not just an academic curiosity but a key factor for strong visual recognition and reasoning. By formalizing the problem, this research opens the door to building AI systems that can better understand complex scenes, potentially improving everything from autonomous driving to medical imaging. The work underscores that current deep learning models still lack a fundamental cognitive ability that humans excel at.