New KL-regularized alignment method boosts medical LVLM accuracy by penalizing ungrounded responses

Medical Large Vision-Language Models (LVLMs) show promise in diagnostics but often hallucinate or misalign with clinical facts. A new paper from researchers at multiple institutions (including University of British Columbia and Vector Institute) identifies three critical flaws in current post-training alignment methods like Direct Preference Optimization (DPO). First, sequence-level reward signals treat clinically critical tokens identically to generic filler text, failing to prioritize diagnostically crucial information. Second, reliance on static supervised fine-tuning (SFT) references as preferred responses introduces an off-policy distribution shift, steering optimization toward stylistic artifacts over clinical correctness. Third, existing alignment objectives lack explicit visual grounding constraints, leaving models insensitive to subtle yet decisive pathological features.

To overcome these, the authors introduce a bidirectional token-wise KL regularizer paired with a visual-contrastive grounding objective. They construct preference pairs by minimally editing model-generated outputs, correcting only clinically erroneous spans while preserving the original linguistic style. The visual-contrastive component uses clean and lesion-corrupted image pairs to penalize responses generated without adequate visual evidence. Extensive experiments across medical imaging tasks and clinical text generation benchmarks validate the approach, showing significant improvements in reducing hallucinations and improving factual consistency. This work represents a step toward clinically reliable LVLMs that can be trusted with read-world diagnostic support.