New study: Knowledge graph reduces LLM hallucinations in clinical trial summaries
GPT-4o, Claude Sonnet, Gemini Flash all fail on unsupported claims in medical summaries.
A new arXiv paper (arXiv:2607.09932) by Robert Williams tackles the critical issue of LLM hallucinations in clinical trial summaries, where errors could mislead healthcare providers, patients, and payers. The study introduces a six-dimension faithfulness annotation schema and tests it on 200 stratified trials from the AACT database, generating 1,800 audience-specific summaries using GPT-4o, Claude Sonnet 4.6, and Gemini 2.5 Flash. The dominant failure mode across all models was 'Unsupported Claims,' with a mean annotation score of just 1.55 out of 3, highlighting the persistent risk of fabricating medical information.
To address this, the author developed a knowledge-graph-augmented retrieval system that integrated structured trial data. Evaluated via cross-encoder NLI, the system produced statistically significant improvements: entailment rose by +0.0125 and overall faithfulness by +0.0130 (p<0.0001). Interestingly, improvement pathways were model-specific—GPT-4o gained primarily by reducing contradictions, while Claude Sonnet 4.6 and Gemini 2.5 Flash improved through increased entailment. This research provides both a rigorous benchmark and a practical mitigation strategy for deploying LLMs in high-stakes clinical contexts.
- Benchmark tested GPT-4o, Claude Sonnet 4.6, and Gemini 2.5 Flash across 1,800 clinical trial summaries using a six-dimension faithfulness schema.
- Unsupported Claims was the worst failure mode for all models, averaging only 1.55 out of 3 on annotation scores.
- Knowledge-graph-augmented retrieval boosted NLI faithfulness scores by +0.0130 (p<0.0001), with model-specific improvement patterns.
Why It Matters
LLMs risk spreading false medical info; this benchmark and fix make clinical summaries safer for providers and patients.