Researchers find LLMs fail introspection tests, debunking prior claims

A team of researchers—Shashwat Singh, Tal Linzen, and Shauli Ravfogel—has published a reality-check paper on arXiv questioning whether large language models (LLMs) can genuinely introspect. Recent studies had argued that models like GPT-4 and Claude can detect and report their own internal states. But this new analysis warns that those conclusions may be premature, drawing lessons from human metacognition research to argue that behavioral evidence alone is insufficient to prove introspection.

The authors re-examine two evaluation paradigms. In the first, models were supposed to detect if their internal states had been tampered with. However, the team found that models couldn't reliably distinguish internal interventions from input manipulations—suggesting they were just spotting anomalies in general. In the second paradigm, models predicted labels derived from their own hidden states. Classifiers with only input access matched the model's in-context predictions, indicating no privileged access. A relabeled control, where semantics were removed, saw performance drop to near chance. The paper concludes that current evidence falls short of proving metacognitive monitoring in LLMs.