HypEHR: Hyperbolic Modeling of Electronic Health Records for Efficient Question Answering

HypEHR, developed by Yuyu Liu, Sarang Rajendra Patil, Mengjia Xu, and Tengfei Ma (accepted at Findings of ACL 2026), tackles electronic health record (EHR) question answering with a radically efficient approach. Traditional LLM-based pipelines are expensive to deploy and fail to exploit the inherent hierarchy in clinical data—such as the ICD ontology (a standard medical classification system) and patient visit trajectories. The team observed that medical ontologies and patient histories naturally exhibit hyperbolic geometry, where tree-like structures can be embedded with low distortion. HypEHR leverages this by using a Lorentzian model (a type of hyperbolic embedding) to represent diagnoses, visits, and questions in a curved space. It answers queries via a geometry-consistent cross-attention mechanism with type-specific pointer heads, which predict the most relevant medical codes or text spans.

The model is pretrained on two tasks: next-visit diagnosis prediction and hierarchy-aware regularization that aligns its representations with the ICD hierarchy. This ensures HypEHR understands not just flat codes but their relationships (e.g., 'pneumonia' being a subtype of 'respiratory disease'). On two MIMIC-IV-based EHR-QA benchmarks (standard datasets for clinical question answering), HypEHR approaches the performance of LLM-based methods while using far fewer parameters—making it significantly cheaper to train and run. The code is publicly available on GitHub, enabling further research and adoption. This work highlights a promising direction: instead of scaling up general-purpose LLMs, specialized geometric models can achieve competitive accuracy with a fraction of the compute, especially in structured domains like healthcare.