ViSAE decodes Vision Transformers with 20x better concept coverage

Vision Transformers (ViTs) achieve high accuracy but often rely on misleading shortcuts (spurious cues), making them risky for real-world deployment. A new approach, ViSAE, draws inspiration from neuroscience to create a mechanistic interpretability toolbox that decomposes ViT internal representations into human-understandable concept circuits. The system uses a probing suite of 64,000 images paired with a 16,000-concept vocabulary, boosting concept coverage efficiency by 20x over ImageNet and improving interpretation accuracy by 28.7% compared to existing concept sets.

ViSAE includes top-down concept reading and bottom-up circuit tracing algorithms that automatically reconstruct the reasoning pathways inside a ViT. Researchers then applied concept editing to steer model behavior—on the WaterBirds dataset, ViSAE improved worst-group accuracy by 48.2%, outperforming prior methods by 23.8%. The work was accepted at ICML 2026 and is fully open-source, offering a scalable way to audit and correct bias in vision models before deployment.