Study finds text prompts override visual cues in medical AI models

A new study accepted at CVPR 2026's MMFM-BIOMED workshop reveals a critical weakness in multimodal large language models (MLLMs) used for medical diagnosis. Researchers tested FundusExpert-1B, an open-source ophthalmology model, on a hemorrhage vs. drusen discrimination task using the BRSET dataset. They found that while the model retains coarse spatial grounding, one-shot textual prompts significantly bias predictions toward the prompted finding. Most alarmingly, when an overlaid lesion contour was paired with an inconsistent textual claim, the text prompt overrode the correct visual cue. Overall accuracy dropped from 75% (visual-only) to 46% when misleading text was added, and Chain-of-Thought reasoning made things worse rather than correcting the error.

These findings have major implications for clinical AI deployment. The authors argue that prompting strategies—often used as a cheaper alternative to fine-tuning—are insufficient for safe medical use, as they introduce diagnostic bias that text-based reasoning cannot self-correct. While limited to a single model and dataset, the study highlights a fundamental flaw: MLLMs may prioritize textual information over reliable visual evidence, a dangerous tendency in high-stakes domains like ophthalmology. The work calls for robust verification mechanisms before such models are trusted in clinical settings.