CLIR-Bench: New benchmark tests AI on irregular clinical time series QA

CLIR-Bench addresses a critical gap in clinical AI evaluation. Existing benchmarks focus on regularly sampled time series or static medical QA, ignoring the reality of ICU data: sparse, irregularly sampled, and asynchronous measurements. The benchmark is constructed from de-identified ICU records through a principled four-stage pipeline, yielding 6,600 QA instances spanning 11 clinical variables (e.g., heart rate, blood pressure). Questions are organized into four capability dimensions (e.g., retrieval, reasoning) and 11 tasks, with each question linked to explicit temporal evidence and derivation rules. This allows evaluation of both answer accuracy and evidence use.

Experiments reveal that current generalist models—including large language models and multimodal systems—perform poorly on these irregular time series QA tasks. They struggle to retrieve relevant sparse evidence and reason over non-uniformly sampled data, often failing to ground answers in the actual temporal observations. The authors argue that stronger methods for irregular time series reasoning are urgently needed. CLIR-Bench provides a rigorous testbed to drive progress in this area, with code and data publicly available to the research community.