Mensing et al. build first joint latent diffusion model for MRI and patient data

Daniel Mensing and colleagues from the University of Bremen and Fraunhofer MEVIS introduce a latent diffusion model that jointly synthesizes volumetric brain MRI and mixed-type tabular clinical data (age, sex, body measurements, ethnicity) in a single framework. The model uses a variational autoencoder to fuse both modalities into a shared latent space, then applies diffusion-based synthesis with cross-attention to ensure coherence. Separate decoders reconstruct the MRI volumes and tabular features from the joint representation, enabling modality-appropriate outputs that are mutually consistent—e.g., generating an MRI with brain anatomy that matches the synthesized patient's age and BMI.

Evaluated on the German National Cohort (NAKO), which includes over 10,000 participants, the generated MRI volumes show high anatomical plausibility, validated by Fréchet distance and precision-recall metrics. For the tabular modality, the model outperforms CTGAN and achieves results comparable to TVAE across standard metrics. The authors claim this is the first demonstration of joint multimodal latent diffusion for MRI and clinical data, positioning it as a key step toward creating digital twins in healthcare—detailed synthetic patient representations that could accelerate research and personalized medicine.