LoFi: Location-Aware Fine-Grained Representation Learning for Chest X-ray

A team of researchers has introduced LoFi (Location-aware Fine-grained representation learning), a breakthrough AI framework designed to overcome critical limitations in medical image analysis. Current contrastive models lack region-level supervision, while large vision-language models often fail to capture fine-grained details in external validation, leading to suboptimal performance in tasks like retrieving similar X-rays or grounding specific phrases to image regions. LoFi addresses this by jointly optimizing three distinct losses—sigmoid, captioning, and a novel location-aware captioning loss—using a lightweight large language model. This multi-task approach enables precise region-level supervision through grounding and dense captioning objectives, fundamentally improving how the AI learns representations of spatially confined, clinically relevant findings.

Building on these enhanced representations, the researchers integrated a fine-grained encoder into a retrieval-based in-context learning pipeline. This architecture allows the system to better understand and localize anomalies within chest X-rays by learning from context provided by similar retrieved images. Extensive validation on two major public datasets, MIMIC-CXR and PadChest-GR, demonstrated that LoFi achieves state-of-the-art performance in both image-text retrieval and the challenging task of phrase grounding, where specific descriptive text must be accurately linked to its corresponding region in the scan. The work, detailed in the arXiv preprint 2603.19451, represents a significant step toward more interpretable and precise AI assistants for radiology.