New Fusion Framework Boosts Time-to-Event Prediction by 5.4%

Accurate time-to-event (TTE) prediction from multimodal clinical data is notoriously difficult due to modality imbalance and distribution shift. In a new preprint on arXiv, Zhang et al. propose a generalizable framework that leverages domain-specific foundation models to align representations from CT imaging and longitudinal electronic health records (EHR). The method encodes each modality independently, then maps them into a shared latent space using four fusion strategies: late fusion, contrastive alignment, cross-attention, and co-attention. The team tested their approach on two clinically distinct tasks—pulmonary embolism (PE) mortality and major adverse cardiovascular events (MACE)—using large, multi-institutional cohorts (PE: 3,099 training, 1,098 internal, 435 external; CVD: 2,951 training, 837 internal, 682 external).

Fusion consistently outperformed unimodal baselines, improving concordance index by 1.5% to 5.4%. Contrastive fusion with CLMBR representations delivered the most consistent and statistically robust gains, particularly for PE mortality prediction. For MACE, cross-attention with one-hot features achieved the best internal performance, while image-guided co-attention excelled externally. The authors emphasize that no single fusion strategy works universally—task-aware selection is critical. This is the first systematic analysis of fusion behavior under modality imbalance in TTE prediction, providing a blueprint for scalable, robust clinical AI deployment.