PALM framework improves radiology report generation by aligning pathology semantics

A new paper from independent researcher Xuan Cuong Ngo proposes PALM (Pathology-Aware aLignment for Medical imaging), a framework designed to fix a key flaw in radiology report generation models. Current radiology-adapted vision-language models often use contrastive learning to pair images with reports, but this can wrongly separate unpaired reports that describe the same pathology. PALM avoids that by introducing shared pathology prototypes that act as a clinically meaningful bridge between radiographic evidence and text, letting compatible cases cluster together without being pushed apart. It also adds Masked Evidence Modeling, which masks image regions and learns from the resulting semantic changes to sharpen the image encoder's sensitivity to local abnormalities.

PALM was tested on three public datasets: MIMIC-CXR, IU X-Ray, and MIMIC-ABN. Results show consistent gains in report generation quality and abnormality-focused robustness compared to standard approaches. The work addresses a practical problem: models often generate clinically plausible but unsupported text by relying on pretrained language priors. By aligning visual and textual features around pathology concepts, PALM pushes the decoder to ground each sentence in actual radiographic evidence. While the paper is a single-author arXiv submission and doesn't report absolute metrics, the method is straightforward and could be integrated into existing radiology AI pipelines. The code and data links are pending, but the approach has clear implications for reducing hallucination in automated medical reporting.