MIT study reveals adversarial fine-tuning limits for Vision Transformers

A new paper from MIT researchers (Gao et al.) provides a mechanistic analysis of adversarial fine-tuning for Vision Transformers (ViTs), a critical component in modern multimodal models like VLMs and VLAs. The team adversarially trained ViTs on low-frequency and high-frequency image corruptions (e.g., blurring, sharpening) and examined attention mechanisms, internal representations, and knowledge evolution. Their results show that while fine-tuning improves model performance and certainty on new instances of the same corruption class, these improvements do not generalize to other corruption classes not seen during training. This poses a serious challenge for deploying ViTs in high-risk real-world applications where diverse, unpredictable perturbations occur.

The study also found that adversarial training did not lead to fundamental changes in the sparse representations learned by ViTs, despite observable shifts in visual attention and knowledge evolution across layers. This suggests that current adversarial fine-tuning methods primarily adjust surface-level attention rather than altering the core representational structure. The work underscores the need for new robustness techniques that address cross-corruption generalization and deeper representational changes, especially as ViTs become integral to safety-critical vision systems.