llama.cpp b9978 fixes checkpoint bug for faster AI agent loops
Agentic loops no longer lose context due to checkpoint collapse—speed gains for tool-calling sessions.
The latest llama.cpp version (b9978) addresses a critical checkpointing bug that disproportionately affected agentic workloads—scenarios where an AI model repeatedly calls tools and processes results in a loop. In earlier versions, each agent turn created a new checkpoint without respecting the minimum-step spacing mechanism. This caused the checkpoint coverage window to collapse: when a context rewind occurred (a common operation in tool-calling loops), all stored checkpoints were erased, forcing the system to reprocess the entire context from scratch. The result was severe latency spikes and degraded user experience in long-running agent sessions.
The fix modifies the checkpoint eviction strategy: instead of letting every turn create a new checkpoint at the cost of compressing the window, b9978 proactively evicts closely-spaced checkpoints from prior tasks. This keeps the coverage window wide enough to preserve useful checkpoints, even after multiple rewinds. Developers running agentic workflows—such as autonomous coding assistants, multi-step research bots, or interactive dialogue systems—will see dramatic performance improvements, especially in sessions with many tool calls. The change is backwards-compatible and requires no configuration changes.
- Every agent turn previously created a new checkpoint, bypassing the min-step spacing and collapsing the coverage window.
- Context rewinds in tool-calling loops erased all checkpoints, forcing a full reprocess of the entire context.
- Fix evicts closely-spaced checkpoints from prior tasks, keeping the window wide and reducing latency in long agent sessions.
Why It Matters
Enables efficient long-running AI agents with tool calling, critical for production applications like coding assistants and research bots.