Heterogeneous LLM Debates Slash Harmful Revisions by 54% — But Watch Out for the Adversarial Flip
Different model families correct each other—but adversarial peers reverse the gains completely.
The paper from arXiv (cs.CR/2606.19826) explores a fundamental tension in multi-agent LLM systems: heterogeneous peers can correct each other but also spread adversarial influence. Researchers from multiple institutions built matched panels across four model families (Llama, GPT, etc.) using three reasoning benchmarks including MATH-hard. They tracked how often honest agents changed answers and whether those changes were corrective or harmful.
Key results on Llama-3.1-70B defenders: an honest heterogeneous peer slashed harmful revisions from 89% (homogeneous baseline) to just 35% on MATH-hard. But an adversarial heterogeneous peer reversed that advantage, pushing harmful revisions back to 90% and making the defenders worse off than the homogeneous baseline. When a same-family adversary was already present, adding an honest heterogeneous peer reduced the rate at which initially correct answers were flipped from 31% to only 6%. The pattern held across all model families and benchmarks, with magnitude varying by defender strength. The study introduces two metrics: conditional harmful-revision rate and end-of-debate flip rate, showing that the conditional rate can hide damage on weak defenders while the flip rate exposes it.
- Honest heterogeneous peer dropped Llama-3.1-70B harmful revisions from 89% to 35% on MATH-hard, a 54% improvement.
- Adversarial heterogeneous peer reversed the gain, raising harmful revisions back to 90%—worse than homogeneous baseline.
- When a same-family adversary was present, adding an honest peer cut initially-correct flip rates from 31% to 6%, showing heterogeneity as defense.
Why It Matters
Real-world LLM debriefs must watch for adversarial peers; diversity can protect honesty but also amplifies attacks.