Debate barely beats random masks in new AI oversight study

A new AI safety paper revisits the classic sparse-pixel experiment from AI Safety via Debate (2018), pitting two oversight protocols—debate and consultancy—against each other. In this setup, a weak judge classifies an image from only 4 of 36 cells, while powerful agents (using exact minimax against the judge) choose which cells the judge sees. The researchers improved on the 2018 experiment by sweeping agent optimization pressure with quantilizers, training judges on-policy, and comparing against random selection with a fixed budget. The goal: isolate whether adversarial debate truly catches misleading evidence better than a single consultant.

The results are surprising. Against a frozen judge, consultancy's selection accuracy degrades as the agent becomes more capable, while debate stays flat—but debate barely surpasses an ensemble of the judge over random masks. That suggests the adversary acts mainly as a regularizer against reward hacking, not through the computational-complexity mechanism originally proposed. With trained judges, debate and consultancy become roughly equally effective. Based on these findings, the authors propose design principles that generalize: train judges on exactly the quantity the protocol aims to produce, let both debaters propose the same answer, and adopt a 'last-' rule where the consultant commits its class only after the full reveal—avoiding uninformative evidence without making criticism free.