AAFLOW+ slashes multi-agent LLM costs by 7.63x with zero-copy KV cache sharing

Multi-agent LLM systems often require agents to repeatedly recompute shared context through expensive prefill, creating a bottleneck. AAFLOW+, introduced by researchers from multiple institutions, solves this by making KV cache a first-class distributed systems object. It builds communication-aware graphs that optimize data, prompts, and reusable model state, providing operators for KV materialization, transfer, fork, composition, and eviction. This zero-copy orchestration lets agents reuse long context without recomputation, dramatically improving efficiency.

Based on an analytical cost model parameterized by empirical microbenchmarks, AAFLOW+ achieves a 50.2x reduction in time-to-first-token (TTFT) and a 7.63x reduction in multi-agent compute cost at 16-agent scale. It also reduces KV memory usage by 1.72–6.10x and increases throughput by over 7.74x. The key insight: KV transmission outperforms recomputation on networks with moderate to high bandwidth, making shared KV state a powerful alternative to passing text between agents.