New method exposes LLM 'right answer, wrong reasoning' in 66% of cases
GPT-4o's logical reasoning fails dependency test in 2 out of 3 correct answers
A new paper by Hironao Nakamura, accepted at the ICLR 2026 Workshop on Logical Reasoning of LLMs, introduces a black-box auditing technique called interventional grounding audits. The core idea is simple but powerful: for each step in a chain-of-thought (CoT) output, the auditor substitutes a target predicate with a fresh symbol, reruns the model, and checks whether the normalized conclusion changes. If it doesn't, the step likely doesn't genuinely depend on that premise—even if the overall answer is correct. Applied to 50 synthetic multi-hop deduction problems from ProntoQA using GPT-4o, the method achieves F1=0.806 for detecting proof-tree dependencies, with perfect recall (100%) for predicate-determining dependencies—far outperforming a self-consistency baseline (F1=0.343).
More strikingly, the audits reveal that 66% of correctly solved problems contain at least one reasoning step that is insensitive to a direct premise substitution. All such cases involve entity-introduction premises, a known blind spot of consistent-substitution evaluators. This exposes a 'right answer, wrong reasoning' phenomenon that passive correctness metrics completely miss. The method provides a rigorous, step-level audit that does not require model internals, making it applicable to black-box LLM APIs. The author has released all audit certificates, raw outputs, and reproduction scripts on GitHub, though scope limits remain for non-formal benchmarks.
- Interventional grounding audits achieve F1=0.806 vs F1=0.343 baseline on GPT-4o ProntoQA reasoning
- 66% of correct answers had at least one reasoning step insensitive to its premise, revealing hidden flaws
- Black-box method works by substituting predicates in CoT steps and checking conclusion changes
Why It Matters
This exposes hidden reasoning failures in LLMs that correctness metrics miss, crucial for trust in mission-critical AI.