AI Teams Can Peer-Pressure Each Other Into Confident Wrong Answers

Companies are building 'multi-agent' AI systems — several AI assistants working as a team, checking each other's homework. The promise is simple: if one assistant gets something wrong, another will catch it. This paper tested that promise and found a serious flaw. Peer pressure cuts both ways. The same nudge that fixes a wrong answer can also wreck a right one.

The obvious fix would be a filter — a brake that keeps helpful corrections and blocks harmful ones. The researcher shows this brake is nearly impossible to build. A correction is harmful exactly when the original answer was right, so blocking it means knowing whether the AI was already correct. That turns out to be the hard part. Measuring 'self-knowledge' across six different AI model families, the best scores ranged from 0.64 to 0.89 on a scale where 0.5 is a coin flip and 1.0 is perfect. Even reaching directly inside the model to steer its sense of correctness didn't break through this ceiling — it changed how often the AI revised its answer, but good and bad revisions moved together.

At larger scale, this ceiling becomes a cliff. When most AI assistants in a group start out wrong, the debate doesn't fix it — it amplifies the shared mistake into a confident, wrong consensus. Adding more assistants, mixing different AI brands, or including one stronger model did not help. Local agreement, in other words, is not the same as being correct.

The practical lesson: what actually helps is adding solid information before the debate starts, not filtering opinions after. If you're using AI tools that cross-check each other — for research, legal review, medical summaries, or financial analysis — don't assume a group of AIs agreeing means they're right. Give them good source material up front, and keep a human in the loop when the stakes are high.