LLM4EHR aligns clinical time series with EHR events via LLMs
A new foundation model that makes ICU data speak the same language as medical events.
LLM4EHR addresses a key limitation in clinical foundation models: insufficient exploitation of shared temporal structures between clinical events and time-series observations in electronic health records. Built on ICU data, the model pairs a domain-adapted LLM (fine-tuned on medical text) with a transformer-based time-series encoder. A regularised contrastive learning objective forces the embeddings from both modalities to align temporally, so events like medication administration are tightly coupled with vital sign trends.
In experiments, LLM4EHR outperforms existing bespoke supervised models and other foundation models on multiple downstream clinical tasks, including mortality prediction, length-of-stay estimation, and diagnosis classification. Ablation studies confirm the importance of the temporal alignment component. Crucially, the learned time-series embeddings transfer well to new hospital cohorts via k-shot adaptation, requiring only a few labelled examples to maintain strong performance. This makes LLM4EHR a promising step toward generalisable, scalable clinical decision support systems.
- Combines domain-adapted LLM with transformer time-series encoder for EHR data.
- Uses regularised contrastive objective to temporally align clinical events and vitals.
- Achieves competitive performance on ICU downstream tasks with transferable k-shot embeddings.
Why It Matters
Makes clinical AI more adaptable across hospitals by aligning messy EHR data with medical events.