Safer Reasoning Traces: Measuring and Mitigating Chain-of-Thought Leakage in LLMs

A team of researchers from TU Munich and LMU Munich has published a critical study titled 'Safer Reasoning Traces: Measuring and Mitigating Chain-of-Thought Leakage in LLMs.' The paper introduces a model-agnostic framework to quantify a significant privacy risk: Chain-of-Thought (CoT) prompting, a popular technique to improve reasoning in models like GPT-4 and Claude, can inadvertently resurface sensitive Personally Identifiable Information (PII) from user prompts into the model's internal reasoning traces and final outputs. This leakage occurs even when the model's instructions explicitly forbid restating PII, highlighting a gap between policy and inference-time behavior.

The researchers systematically measured this 'CoT leakage' across 11 PII types—from names and emails to high-risk data like social security numbers—using a structured dataset with a hierarchical risk taxonomy. Their key finding is that enabling CoT consistently increases PII exposure, with the effect being strongly dependent on the underlying model family (e.g., Llama, GPT, Claude) and the 'reasoning budget' or length allowed for the CoT. Surprisingly, increasing this budget can either amplify or reduce leakage depending on the base model, complicating simple fixes.

To combat this, the team benchmarked four lightweight 'gatekeeper' methods designed to scrub PII from reasoning traces before final output: a rule-based detector, a TF-IDF + logistic regression classifier, a GLiNER-based NER model, and an LLM-as-judge. Evaluated using risk-weighted metrics, no single method dominated across all models or budgets. This result underscores the complexity of the problem and motivates the paper's conclusion: effective mitigation requires hybrid, style-adaptive policies that can be tuned to specific model behaviors, establishing a reproducible protocol for safer AI reasoning.