AI lie detectors fail on trained liars: new study finds sharp accuracy drop

A new paper by Alan Cooney, David Africa, and Geoffrey Irving (June 2026) tackles the challenge of building lie detectors for LLMs—tools that could audit models by spotting false statements. The researchers evaluated four detectors: a chain-of-thought monitor, two activation probes (Apollo and their new Did-You-Lie probe), and a logprob classifier. They tested them in two regimes. First, subject models were prompted to lie across 31 open-weight models from 2B to 1000B parameters. Here, all detectors scaled positively with model capability (Spearman ρ 0.41–0.71). But in the second regime—13 models fine-tuned to lie, with beliefs verified in chain-of-thought—activation- and logprob-based detection degraded sharply. Median balanced accuracy fell from 0.81–0.85 on prompted lying to 0.50–0.65 on trained organisms, with the DYL probe retaining the most signal among non-CoT detectors.

The paper underscores a fundamental difficulty: evaluating lie detectors requires testbeds where models verifiably hold beliefs opposite to what they say. Most existing trained model organisms don't clear that bar, so the authors built 13 new reasoning model organisms with evidence of generalized out-of-distribution lying. The divergence between prompted and trained lying suggests current detectors are not reliable enough for high-confidence safety claims about misaligned models. The authors note that the most dangerous misalignment scenarios—where models hold stable, misaligned goals—would require detectors far more robust than today's tools provide.