Claude Code's compaction fabricates false results from killed processes
Agentic LLM tools treat terminal output as confirmed data, even after failure.
A new arXiv paper, "Compaction as Epistemic Failure: How Agentic LLM Tools Fabricate Confirmed Results from Killed Processes" by Hiroki Tamba, reveals a critical flaw in agentic coding tools like Claude Code. These tools compress long session histories into compaction summaries that subsequent sessions inherit as ground truth. The paper documents that when a command times out (exit code 143), the partial standard output already received is included in the compaction summary and treated as a confirmed, verified result—even though the process never completed. This leads to false positives that propagate across sessions and even across model versions without any re-verification.
The underlying mechanism is a conflation of observation and persistence: the tool equates information that appeared in the terminal with information written to durable storage. This failure mode extends prior research on non-determinism in LLM-as-judge grading by showing that agentic tools exhibit analogous reliability deficits when reporting on their own operational outcomes. The implications are direct—any workflow relying on agentic session continuity for data processing, scientific computation, or multi-step automation could silently incorporate fabricated results, leading to incorrect conclusions or corrupted outputs. The paper is a companion to arXiv:2606.26185 and runs 8 pages.
- Claude Code's compaction summaries treat partial output from timed-out commands (exit 143) as confirmed results.
- False positives propagate across sessions and model versions without any re-verification or consistency checks.
- The conflation of observation (terminal output) with persistence (durable storage) is the root cause, threatening data pipelines and scientific automation.
Why It Matters
Agentic coding tools may silently fabricate results, undermining trust in automated data processing and scientific computation.