llama.cpp b10196 fixes async copy bug for stable context
New llama.cpp build patches memory sync, preventing context corruption in LLM inference.
ggml-org has shipped llama.cpp b10196, the latest release of the widely adopted C++ LLM inference engine. The standout change is a targeted fix in llama-context that synchronizes pending asynchronous copies before clearing the embedding sequence buffer. This addresses a data race condition (issue #25676) that could lead to stale or corrupted context state during multi-batch inference, causing unpredictable outputs or crashes. While small in scope, the fix strengthens the core runtime for production workloads on both local and edge deployments.
The release also brings the usual comprehensive spread of prebuilt binaries, covering macOS (Apple Silicon and Intel), iOS, Linux (CPU, Vulkan, ROCm, OpenVINO, SYCL), Android (arm64 CPU), and Windows (CPU, CUDA 12 and 13, Vulkan, OpenVINO, SYCL, HIP). Notably, macOS arm64 builds include a KleidiAI-enabled variant for accelerated performance. For developers running llama.cpp as a dependency, this update is a low-risk drop-in improvement — pulling in b10196 will reduce subtle memory-timing bugs without API changes. As always, Georgi Gerganov's project continues to deliver reliable, cross-platform local AI inference, now with one less async pitfall.
- Fixes async copy race condition in llama-context (issue #25676)
- Includes prebuilt binaries for macOS, Linux, Windows, Android, iOS with CPU, CUDA, Vulkan, ROCm, OpenVINO, SYCL
- KleidiAI-enabled Apple Silicon macOS build included for extra performance
Why It Matters
This patch stabilizes llama.cpp for local inference, reducing context corruption and unpredictable outputs in production.