Align then Refine: Text-Guided 3D Prostate Lesion Segmentation
A novel multi-encoder U-Net architecture achieves state-of-the-art results on the PI-CAI dataset by integrating localized text guidance.
A collaborative research team has published a paper titled 'Align then Refine: Text-Guided 3D Prostate Lesion Segmentation,' introducing a novel AI architecture that significantly improves the automated 3D segmentation of prostate lesions from biparametric MRI (bp-MRI) scans. The core challenge in this medical imaging task is achieving high precision by reliably fusing information from multiple MRI modalities while maintaining anatomical consistency. The team's solution is a new multi-encoder U-Net model that incorporates three technical innovations designed to inject fine-grained, lesion-level semantics through text guidance.
The first innovation is an alignment loss that enhances the similarity between text descriptions and image foregrounds to better inject lesion semantics. Second, a heatmap loss calibrates the model's similarity maps and suppresses irrelevant background activations. Finally, a confidence-gated multi-head cross-attention refiner performs localized boundary edits in high-confidence regions. A phase-scheduled training regime stabilizes the optimization of these components. The method consistently outperforms prior approaches, establishing a new state-of-the-art on the benchmark PI-CAI dataset through enhanced multi-modal fusion and precise, localized text guidance. The team has made their code publicly available, facilitating further research and clinical application development.
- Introduces a novel multi-encoder U-Net with three core innovations: alignment loss, heatmap loss, and a confidence-gated cross-attention refiner.
- Achieves state-of-the-art performance on the PI-CAI dataset for 3D prostate lesion segmentation from MRI.
- Enables precise, text-guided localization of lesions, moving beyond general vision-language models to fine-grained medical semantics.
Why It Matters
This advance enables more reliable, automated analysis of prostate cancer in MRI, potentially improving diagnostic accuracy and consistency for radiologists.