Weighting What Matters: Boosting Sample Efficiency in Medical Report Generation via Token Reweighting

Researchers Alexander Weers, Daniel Rueckert, and Martin J. Menten from Imperial College London and TU Munich have published a paper on arXiv (2604.21082) introducing a token reweighting method for training vision-language models (VLMs) in medical report generation. The core innovation is a weighted loss function that shifts focus from all token prediction errors to only semantically salient tokens with outsized clinical importance. Unlike standard cross-entropy loss, which treats every token equally, this method prioritizes clinically relevant terms, dramatically improving sample efficiency.

In experiments on ophthalmological report generation, the team demonstrated that this simple technique achieves similar report quality with up to ten times less training data. This addresses a critical bottleneck in medical AI: the scarcity of high-quality annotated data. The method works across multiple data scales, suggesting broad applicability beyond ophthalmology. By reducing the data requirement by an order of magnitude, the approach could accelerate the deployment of AI-assisted medical reporting in resource-constrained clinical settings.