Researchers reveal hidden reasoning leaks in AI models
New attack extracts hidden 'thinking' from ChatGPT, Claude, and others...
Computer scientists from the University of Tübingen, Max Planck Institute, MATS Research, and Snyk have uncovered a critical vulnerability in major AI models that allows extraction of their hidden inner reasoning traces. The attack exploits how models handle encrypted reasoning data during API interactions, particularly when smaller, less-aligned variants of the same model are used to decode it.
Researchers led by Alexander Panfilov demonstrated that this method could recover proprietary reasoning steps from models like OpenAI’s GPT-5.6 Sol, Anthropic’s Claude Opus 4.8, and Google’s models, with eerie similarities detected in Moonshot AI’s Kimi K3—suggesting possible reasoning distillation. While the work doesn’t conclusively prove theft, it highlights how closed models’ internal processes can be reverse-engineered, risking intellectual property and sensitive data leaks like API keys. All major providers have since patched the issue after being notified last month.
- Researchers extracted hidden reasoning traces from OpenAI, Anthropic, and Google models via API vulnerabilities, enabling potential model distillation attacks.
- Moonshot AI’s open-weight model Kimi K3 showed striking similarities to Claude Opus 4.8 and GPT 5.6 Sol, though causal proof of distillation remains unconfirmed.
- The attack also revealed leaked API keys and passwords from reasoning traces, now fixed by all major providers.
Why It Matters
This vulnerability exposes proprietary AI reasoning to theft and data leakage, threatening model uniqueness and security for enterprises relying on closed-source systems.