Attention Deficits in Language Models: Causal Explanations for Procedural Hallucinations

A new research paper titled 'Attention Deficits in Language Models: Causal Explanations for Procedural Hallucinations' provides a detailed diagnosis of why large language models fail at seemingly simple tasks. The study, authored by Ahmed Karim, Fatima Sheaib, and four colleagues, examines what they term 'procedural hallucination'—when models fail to execute verifiable, prompt-grounded specifications even when the correct value is present in context.

In long-context binding tasks with known single-token candidate sets, the researchers found that most errors are readout-stage routing failures. These decompose into Stage 2A (gating) errors where the model doesn't enter answer mode, and Stage 2B (binding) errors where it enters answer mode but selects the wrong candidate, often due to recency bias. In the hard regime, Stage 2B accounted for most errors across model families tested.

Crucially, on Stage 2B error trials, a linear probe on the final-layer residual stream recovered the correct value far above chance—74% vs. 2% on Qwen2.5-3B—indicating the answer is encoded but not used. The researchers formalized this 'present but not used' phenomenon using available vs. used mutual information and pseudo-prior interventions, creating output-computable diagnostics and information-budget certificates.

Most practically, an oracle checkpointing intervention that restates the true binding near the query nearly eliminated Stage 2B failures at long distances. For Qwen2.5-3B, this intervention improved performance from 0/400 to 399/400 at k=1024 context length. This suggests specific architectural interventions could significantly reduce certain types of hallucinations in production systems.