b8644
The latest commit reverts a problematic quantization change and adds new builds for Windows CUDA 13 and openEuler.
The open-source project llama.cpp, maintained by ggml-org, has pushed a significant new release tagged b8644. The core of this update is a reversion of a previous change (commit 17193cc) that had introduced quantization to the Key-Value (KV) cache used for Sliding Window Attention (SWA). Quantizing this cache was causing degraded model performance and accuracy during inference, making this a crucial stability fix for users leveraging attention-windowed models.
Alongside this bug fix, the release significantly expands the matrix of available pre-built binaries, making it easier to deploy across diverse environments. Newly supported targets include Windows builds with CUDA 13.1 DLLs, providing an upgrade path for users on newer NVIDIA drivers. Furthermore, the team has added multiple builds for the openEuler Linux distribution, covering both x86 and aarch64 architectures with support for Huawei's Ascend AI processors (310p and 910b). This broadens the project's reach into enterprise and edge-computing scenarios.
The commit was automatically generated and signed by GitHub's verified signature system, ensuring its integrity. This release underscores the rapid, community-driven development pace of llama.cpp, which is essential for optimizing and running large language models like Meta's Llama series efficiently on consumer hardware and specialized accelerators.
- Reverts commit 17193cc to fix a bug where quantizing the Sliding Window Attention KV cache hurt performance.
- Adds new pre-built binaries for Windows with CUDA 13.1 DLL support and multiple openEuler (Ascend AI) configurations.
- Maintains the project's wide hardware support across macOS, Linux, Windows, and iOS with CPU, GPU, and accelerator backends.
Why It Matters
This fix ensures more accurate and efficient local AI inference, while expanded builds lower the barrier to deployment on specialized hardware.