Same-model AI agents co-fail 90% of the time, breaking redundancy math

A new arXiv preprint, "Agent Behavioral Contracts II: Certifying Compositional Reliability Without Assuming Independence," by Varun Pratap Bhardwaj, Garima Singh, and Arun Pratap Bhardwaj, directly challenges a core assumption in multi-agent system safety: that component failures are independent. Standard reliability bounds multiply component reliabilities to estimate system-level success, but this is only valid if failures are uncorrelated. The authors preregistered an experiment with 18,000 two-agent handoff missions, scored deterministically, and found that two instances of the same model co-fail on 90.0% of missions when either fails (log OR 6.66, phi 0.916). This positive dependence means redundancy is over-credited—identical agents fail in lockstep, not independently.

Swapping in a different model reduced the co-failure association in all six contrasts, but using a different vendor with a different model did not, a null result reported as registered. The paper then shows that standard fixes fail: the assumption-free bound is often vacuous, and fitting a dependence model becomes provably less reliable as data grows—the identification gap stays O(1) while the bootstrap haircut shrinks at O(n^-1/2), so more data makes the certificate worse. Their solution is a finite-sample certificate using a linear program over a Bonferroni-Clopper-Pearson box. Enriching ten moment functionals to fourteen narrowed the reliability interval by 85.7% and lifted the certified floor from 0.2455 to 0.4116. A companion anytime-valid certificate keeps type-I error at 0.0471 under optional stopping. The authors released code, scoring scripts, and preregistration.