A Randomized Controlled Trial and Pilot of Scout: an LLM-Based EHR Search and Synthesis Platform

A team led by Michael Gao and colleagues at Duke University developed Scout, an LLM-powered platform that allows clinicians to search and synthesize electronic health record (EHR) data using natural language queries. Each response includes citations linking claims to original data sources for easy verification. In a prospective randomized, evaluator-blinded crossover trial across seven clinical specialties (20 participants, 200 structured cases), clinicians performed realistic tasks using either Scout or standard EHR alone. Scout reduced task completion time by 37.6% and significantly decreased perceived workload, with the largest reductions in mental demand, effort, and temporal demand. Non-inferiority analyses confirmed that tasks completed with Scout maintained accuracy, completeness, and relevance compared to EHR-only tasks.

In a concurrent pilot deployment across over 200 users and more than 20 specialties, Scout generated over 6,600 interactions in three months, demonstrating diverse clinical and administrative use cases. The team also evaluated an automated LLM-as-judge framework for error detection, which flagged outputs at low rates. Manual review of a subset revealed that most flagged claims were actually supported by patient charts, underscoring the need for human validation. These findings provide early trial-based evidence that LLM-powered EHR tools can meaningfully reduce clinical administrative workload while maintaining output quality—a critical step toward addressing clinician burnout.