NIH's unified lesion framework beats nnUNet by 28.5% Dice

A team led by Ruida Cheng at the NIH National Institutes of Health (including collaborators from the National Library of Medicine and NIH Clinical Center) has released a unified 2D lesion analysis framework that processes the original DeepLesion dataset end-to-end. Unlike prior work that required separate models for detection, segmentation, and reporting, this single framework integrates LLM-based reasoning, bounding box detection, segmentation, and radiology report generation. In testing, it achieved a lesion detection mAP50 of 70.1% and mAP50-95 of 46.4%, while segmentation reached a Dice score of 62.6%—a 28.5% improvement over the nnUNet baseline on the challenging original DeepLesion dataset. The framework also generates short reports with BLEU-1 of 64.3%, BLEU-4 of 49.6%, METEOR of 34.7%, and ROUGE-L of 60.1%, incorporating spatial and anatomical context for more clinically relevant text.

The authors note that previous iterations used the ULS23 dataset with short-form findings, but this work directly uses the original DeepLesion dataset to tackle the notoriously difficult segmentation task. By unifying detection, segmentation, and report generation in a 2D framework, the approach reduces the need for complex 3D pipelines while maintaining strong performance. The implementation, dataset, and pre-trained models are publicly available on GitHub, making it reproducible for the medical imaging community. This could accelerate clinical workflow by providing radiologists with a single tool that localizes lesions, outlines their boundaries, and drafts descriptive reports—potentially reducing manual annotation overhead and improving consistency in lesion tracking and documentation.