E-MRL reduces hallucinations in 3D tumor analysis via evidence grounding

Vision-language models (VLMs) used for volumetric medical report generation often hallucinate—producing seemingly correct diagnoses that rely more on language patterns than actual visual evidence from 3D CT scans. Current Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL) methods primarily optimize text fidelity, inadvertently rewarding correct guesses derived from linguistic priors rather than genuine perception. To tackle this, Sijing Li et al. propose E-MRL (Evidence-driven Multimodal Reinforcement Learning), which reformulates report generation as a Markov Decision Process of “diagnosis-localization-verification.” The model is explicitly trained to identify a key evidence slice alongside the global diagnostic report, grounding its findings in a verifiable visual anchor. A novel cross-view consistency reward validates the semantic alignment between the golden-standard report and a local visual re-query of that key slice, providing additional reinforcement for correctly localized reasoning.

Experiments on large-scale 3D CT tumor datasets demonstrate that E-MRL outperforms both SFT and RL baselines in reducing visual hallucinations and boosting diagnostic accuracy. By enforcing cross-view alignment, the method ensures that the model’s conclusions are backed by actual image evidence rather than textual shortcuts. This approach not only improves reliability but also increases clinical interpretability—doctors can inspect the specific slice that drove the diagnosis. The paper (arXiv:2606.23888) showcases a path toward more trustworthy AI in radiology, potentially reducing false positives and improving patient outcomes. The 9-page paper includes 2 figures and is authored by researchers from multiple institutions.