Handling and Interpreting Missing Modalities in Patient Clinical Trajectories via Autoregressive Sequence Modeling

A team of researchers has published a new paper proposing a novel AI framework to tackle a persistent problem in healthcare machine learning: handling missing data modalities in patient records. Clinical datasets are inherently messy—patient histories are temporal, and different tests (like lab results, imaging, notes) are often collected at different times or not at all. The researchers, Andrew Wang, Ellie Pavlick, and Ritambhara Singh, reframe the diagnostic challenge as an autoregressive sequence modeling task, similar to how large language models (LLMs) predict the next word in a sentence. Their model uses causal decoders to predict a patient's future clinical state based on their past multimodal trajectory.

The framework employs a two-stage training process. First, it uses a novel "missingness-aware contrastive pre-training" objective. This technique learns to integrate information from various data types—even when some are absent—into a cohesive, shared representation. Second, this pre-trained model is fine-tuned on specific tasks using transformer architectures. The approach demonstrated superior performance on major healthcare benchmarks like MIMIC-IV and eICU. Crucially, the team applied interpretability techniques to show that their pre-training method successfully mitigates the divergent and unreliable behavior that typically occurs when models encounter missing data, moving a step closer to transparent and safe clinical AI.