FairGlucose benchmark exposes hidden CGM accuracy gaps across patient subgroups

A team of researchers led by Junjie Luo has introduced FairGlucose, a fairness benchmark for continuous glucose monitoring (CGM) AI systems, revealing that population-level validation masks significant accuracy disparities across patient demographics. The cohort includes 300 patients balanced across 12 demographic strata (age × gender × type 1/type 2 diabetes), generating 132,480 forecasting samples and 3,945 logged behavioral events (meals, exercise, medication) from 81 patients. Benchmarking 33 models across four model families on 2-hour glucose forecasting, the team found that aggregate out-of-distribution metrics appear stable — around 1.0 — yet subgroup-level ratios range from 0.8 to 1.4. Most striking: Type 1 diabetes patients experience 6 mg/dL higher prediction error than Type 2 patients (p < 0.001), and this disparity persists across all 33 models, suggesting it is a property of the prediction task itself rather than a flaw of any single architecture.

Further analysis shows subgroup performance gaps correlate with the proportion of clinically hard cases, and input-length sensitivity varies across demographics — indicating that personalized model configurations are needed. Notably, frontier LLMs underperform specialized neural models by 1–6 mg/dL, while behavioral event data contributes negligibly (~0.1 mg/dL) even under oracle event access. The authors argue these findings demonstrate that population-level external validation alone is insufficient for equity assessment in digital health AI. They call for subgroup-disaggregated reporting to become the default standard in clinical AI validation, a move that could reshape how CGM-based tools are evaluated before deployment.