Grad-CAM for Vision Transformers: new audit finds 175 papers lack rigor

Gradient-weighted Class Activation Mapping (Grad-CAM) is a cornerstone technique for visualizing why deep learning models make decisions. But it was built for convolutional neural networks, where spatial feature maps and channel dimensions have clear meanings. Vision Transformers (ViTs) break that assumption by representing images through tokens, attention mechanisms, residual streams, and multimodal interactions. Applying Grad-CAM to ViTs therefore requires nontrivial choices — yet many papers treat it as a trivial extension.

The new arXiv study (2608.05258) systematically reviews this gap. Screening over 550 papers, the authors identified 175 that apply Grad-CAM or adjacent methods to ViTs. Their audit reveals that most publications omit a full mathematical or implementation-level account of how Grad-CAM is adapted to transformer representations. To address this, they introduce a descriptive taxonomy that makes explicit four key dimensions: feature locations, gradient targets, spatial reconstruction steps, and aggregation choices. The taxonomy doesn't prescribe one correct approach but maps the range of methodological decisions — giving researchers a common language to report precisely what they did, and helping the community assess rigor, reproducibility, and interpretability of explainability results in ViT-based systems.