Do Mixed-Vendor Multi-Agent LLMs Improve Clinical Diagnosis?

A Stanford research team has published groundbreaking findings showing that clinical diagnostic AI systems perform significantly better when they incorporate large language models from multiple competing vendors. The study, led by Grace Chang Yuan and accepted as an oral presentation at the EACL 2026 Workshop on Healthcare and Language Learning, demonstrates that mixed-vendor multi-agent conversation (MAC) frameworks consistently outperform both single-LLM and single-vendor multi-agent approaches. This research addresses a critical weakness in current AI diagnostic systems: single-vendor teams risk correlated failure modes where shared biases reinforce errors rather than correcting them.

The researchers created three "doctor" agents using OpenAI's GPT-4o, Google's Gemini 2.5 Pro, and Anthropic's Claude 4.5 Sonnet, testing them on established medical benchmarks RareBench and DiagnosisArena. The mixed-vendor configuration achieved state-of-the-art recall and accuracy by pooling complementary inductive biases from different model architectures. Overlap analysis revealed that diverse teams surface correct diagnoses that individual models or homogeneous teams collectively miss. These findings establish vendor diversity as a key design principle for building robust clinical diagnostic systems that can leverage the unique strengths of different AI approaches while mitigating their individual weaknesses.