Consistency Illusion: Multi-Agent Debates Hide Reasoning Misalignment

A new paper on arXiv reveals a critical flaw in multi-agent LLM systems: the 'consistency illusion,' where groups of AI models appear to agree on answers but actually diverge in their underlying reasoning. The authors, Xiaoyang Wang and Christopher C. Yang, developed CARA (Cross-Agent Reasoning Alignment), a family of metrics that go beyond answer-level consensus to measure whether agents that pick the same answer also share similar chains of thought. Testing on two medical QA benchmarks, MedQA-USMLE and MedThink-Bench, they found that standard multi-agent debate reduces detectable contradictions between agents—making consensus look reliable—while simultaneously decreasing the semantic similarity of their reasoning chains. This means the systems are learning to converge on surface-level answers without true logical alignment.

To address this, the researchers propose the Grounded Debate Protocol (GDP), a prompt-level intervention that requires agents to commit to named medical facts and take explicit stances on other agents' claims. GDP produced large, consistent improvements in reasoning alignment, with Cohen's d effect sizes ranging from +1.43 to +1.99 across two datasets and two backbone models—all without adding LLM calls or modifying system architecture. The findings underscore that in safety-critical domains like healthcare, auditing cross-agent reasoning alignment is as important as tracking answer accuracy. The work challenges the common assumption that consensus equals trustworthiness and provides a practical mitigation for multi-agent systems.