MedDiT4SR: Tri-Stream Adaptation Boosts Medical Image Super-Resolution
New tri-stream framework adapts diffusion transformers to sharpen medical scans without hallucinated details.
Medical image super-resolution (MedSR) aims to recover fine anatomical structures from degraded observations without introducing unsupported details from generative priors. Large-scale pre-trained multimodal diffusion transformers offer strong visual priors, but adapting them to MedSR is nontrivial. Conventional ControlNet-style adaptation treats the low-resolution (LR) image as an external condition injected via one-way connections, preventing joint updates with denoising and semantic representations.
MedDiT4SR introduces a tri-stream framework that integrates LR, noisy latent, and text representations directly into multimodal diffusion-transformer blocks, enabling joint evolution of all streams. To complement global token interaction, a Super-Resolution Adapter aggregates scale-dependent local tokens while suppressing interpolation-induced redundancy. A Semantic Alignment Refiner further calibrates local LR responses using prompt-conditioned semantic information. Experiments under in-domain and within-modality cross-dataset settings validate the approach across diverse imaging domains, signaling a practical leap for clinical image enhancement.
- Tri-stream architecture integrates LR, noisy latent, and text representations within diffusion transformer blocks.
- Super-Resolution Adapter aggregates scale-dependent local tokens and reduces interpolation redundancy.
- Semantic Alignment Refiner boosts accuracy by conditioning local LR features on prompt semantics.
Why It Matters
Enables sharper, more faithful medical image reconstruction, improving diagnostic accuracy without introducing artifacts.