ARAT Framework Reveals 'Correlated Agreement Blindness' in Multi-Agent AI

A new paper accepted at PAAMS 2026 introduces ARAT (Arbitrated Reasoning Agents for Alarm Triage), a system designed to expose and mitigate a structural blind spot in multi-agent triage: correlated agreement blindness. The researchers, led by Shay Seiya McDonnell, demonstrate that as base learners improve and become more accurate, they also tend to converge in their predictions. This convergence weakens safety monitoring that relies on disagreement-triggered escalation—because when all agents agree, even on wrong answers, the system often fails to escalate. Using 82,332 holdout samples from the UNSW-NB15 network intrusion detection dataset, they found that 57.2% of all errors occur when agents agree, and a staggering 90.6% of dangerous under-predictions (missed attacks) evade disagreement-based monitoring, even after conservative override.

ARAT is a directed-star architecture combining an inductive Random Forest agent, an analogical case-based k-NN agent, and a calibrated meta-model that arbitrates triage decisions. The system introduces two key mechanisms: a conservative override that flags high-risk agreements, and a safety-flag gate that further filters outputs. Combined, they reduce under-prediction rates from 4.80% (soft voting) to just 1.70%, a 3.1 percentage point improvement. Ablation studies confirm that strengthening individual base learners actually increases error correlation while reducing disagreement, directly creating the blind spot. Cross-dataset validation on clinical readmission (predicting patient readmissions) supports the same pattern, suggesting the risk is widespread. The authors warn that as multi-agent pipelines deploy increasingly capable and correlated models, reliance on disagreement as a safety signal becomes dangerously unreliable.