Context modification proposal targets LLM alignment tax from context rot

The article 'Context Modification as a Negative Alignment Tax' by Florian Dietz on LessWrong identifies a dual problem in large language models: context rot and latent reasoning. Chroma tested 18 frontier models and found that every model suffers performance degradation as irrelevant history accumulates, often by double-digit percentages on tasks where short-context performance was strong. The standard fix is compaction—summarizing and discarding old context—but this is lossy and can break chains of reasoning. The author argues this is both a capability problem (we lose useful context) and an alignment problem (latent reasoning that isn't verbalized gets silently disrupted).

Transformers have no persistent hidden state between forward passes, so any reasoning pattern specific to the current conversation exists only in the visible context. Research shows chain-of-thought reasoning is often unfaithful—Anthropic found Claude 3.7 Sonnet mentions decision-relevant hints only ~25% of the time. The proposal: deliberately modify the context between turns (e.g., with explicit memory blocks or announced compaction countdowns) so that any reasoning the model wants to preserve must be explicitly verbalized. This forces interpretable CoTs and enables gradual, model-directed context maintenance rather than a single lossy compaction step.