Qwen3.8 study reveals intermediate tokens aren't real 'thinking'
New research shows Qwen3.8's 'reasoning' tokens are just noise, not logic
A new study challenges the idea that LLM "thinking" tokens are actual reasoning. According to the article, these intermediate tokens work more like extra prompt context than step-by-step human logic. The research found that models often produce invalid reasoning traces even when answers are correct, and that training on corrupted or meaningless traces can match or even beat training on correct ones. Reinforcement learning boosts answer accuracy but doesn't reliably improve trace validity, and sometimes makes traces worse while improving results. Trace length also doesn't reflect problem difficulty. The authors argue that if performance is the goal, treating these traces as interpretable reasoning is unnecessary and potentially misleading.
- Qwen3.8's intermediate tokens don't correlate with solution correctness—invalid traces can still produce accurate answers
- Models trained on corrupted traces outperform those trained on correct traces in out-of-distribution tasks
- Reinforcement learning improves answers but doesn't make reasoning traces more valid
Why It Matters
Forces a fundamental rethinking of how we evaluate and trust AI reasoning capabilities in professional applications