Gemini LLMs detect EHR documentation inconsistencies in 70% of admissions

A team led by Jian Lu and colleagues at Vanderbilt and Duke University Medical Centers has published a formative study exploring how general-domain large language models (LLMs) can detect internal documentation inconsistencies in electronic health records. Their pipeline uses Gemini 2.5 Pro for open-ended candidate identification, followed by Gemini 2.5 Flash for context-grounded verification. Applied to 3,000 randomly sampled MIMIC-IV-Note discharge summaries, the system identified 3,460 candidate inconsistencies—affecting 69.7% of admissions. Examples included mismatched demographics, conflicting allergy lists, procedure-date errors, and medication discrepancies, all with direct implications for clinical reasoning and patient safety.

Manual review by clinical experts revealed recurring failure modes. The LLMs struggled when verification required temporal reasoning (e.g., tracking changes over a hospital stay), understanding evolving diagnoses, or knowledge of outpatient-prescribing conventions. The authors propose a graded ontology spanning strict contradiction and ambiguity, characterizing each case by category, section, domain, and inconsistency axis. This work establishes a methodological foundation for large-scale, validated EHR inconsistency analysis, though reliability at scale will require specialized context-aware models.