The Heatmaps Doctors Use to Trust Breast Scan AI May Be Misleading

AI is increasingly used to help radiologists spot cancer in breast MRI scans. When the software flags something suspicious, doctors want to know why — so researchers generate "saliency maps," colorful heatmaps that highlight the parts of the image the AI supposedly relied on. These pictures look convincing. But looking convincing is not the same as being accurate, and that gap is exactly what this study went after.

The team trained a Vision Transformer (a popular type of AI image model) on a public breast MRI dataset, then tested seven different heatmap methods against it. Their first finding was unsettling: which method comes out on top depends heavily on how you probe the AI. Blur parts of the image and one method wins; hide the model's attention instead and the rankings flip. In other words, a heatmap can look great simply because of the test you chose.

The second problem was subtler. Some heatmaps answer "this looks like cancer," while others only answer "something interesting is here." Comparing those two side by side is unfair, like judging a weather forecast against a coin flip. The researchers built matched versions of the methods so each could be judged on the question it actually answers.

When tested fairly, Grad-CAM and Gradient Attention Rollout were the strongest at pointing to cancer-specific evidence. The honest catch: this is a single-dataset research study, not a hospital-ready tool. It offers doctors no final verdict on which heatmap to trust. What it does show is that the explanations we use to justify medical AI are shakier than they appear — a warning worth heeding before anyone signs off on AI reading scans unsupervised.