Agent Frameworks

RCWT: LLM performance crashes when coordination tokens crowd task data

GPT-4.1-mini, Claude Haiku 4.5, and Gemini 2.5 Flash all degrade sharply when coordination content pushes task evidence below a few hundred tokens.

Deep Dive

Multi-agent and memory-augmented LLM systems often cram coordination content—shared state, prior discussions, tool outputs, summaries, role instructions—into the same finite prompt as the current task. This creates an allocation problem: every token spent on coordination is unavailable for task instructions or evidence. The new RCWT protocol systematically measures this task-budget displacement effect by varying coordination content while controlling total budget, position order, task family, and scoring. In the main context-dependent recall task at a window size of 4096, three commercial models (GPT-4.1-mini, Claude Haiku 4.5, Gemini 2.5 Flash) remain near baseline through moderate overhead, then degrade sharply once residual reference evidence falls to a few hundred tokens. Window-scaling summaries align with a task-specific residual-budget interpretation rather than a fixed percentage threshold, though the authors caution this is descriptive, not a universal law.

To test whether the fixed-budget cliff stems from coordination volume itself or simple displacement, the researchers added an intact-task ablation: the full task and reference block remains present while coordination tokens increase by expanding total prompt length. In that setting, every tested call returned all scored fields correctly, even at a 95% coordination ratio. This narrows the claim: the main RCWT cliff is best read as task-budget displacement, not proof that coordination volume alone causes semantic interference. RCWT is thus a measurement primitive for context-allocation budgeting, not a complete theory of multi-agent benefit or session-level coordination. The paper provides a practical diagnostic for developers building agent workflows, helping them anticipate when adding chat history or tool outputs will trigger a sudden accuracy drop.

Key Points
  • At W=4096, GPT-4.1-mini, Claude Haiku 4.5, and Gemini 2.5 Flash all degrade sharply when residual task evidence falls below ~200 tokens.
  • An ablation keeping the task block intact shows zero performance loss up to 95% coordination ratio, confirming the cliff is due to budget displacement, not semantic interference.
  • RCWT is a measurement primitive for context-allocation budgeting, not a full theory of multi-agent benefit or session-level coordination.

Why It Matters

Helps AI engineers allocate context windows efficiently in multi-agent systems to avoid sudden accuracy cliffs.

📬 Get the top 10 AI stories daily