llama.cpp b9776 fixes Vulkan overflow with bias-before-softmax patch
118k-star GitHub project ships critical flash attention overflow fix for Vulkan GPUs
The latest release of llama.cpp, tagged b9776, addresses a critical numerical stability issue in the Flash Attention implementation for Vulkan compute shaders. By reordering the bias addition before the softmax operation, the patch prevents overflow errors that could corrupt inference on AMD and other Vulkan-compatible GPUs. This fix is especially relevant for users running large 70B+ parameter models where attention scores are large.
The release was published by the ggml-org maintainers and includes expanded build matrix support: macOS Apple Silicon (both with and without KleidiAI acceleration), iOS XCFramework, Linux across x64/arm64/s390x with CPU, Vulkan, ROCm 7.2, OpenVINO, and SYCL FP32/FP16, Windows with CUDA 12/13, OpenCL Adreno, Vulkan, OpenVINO, and HIP, plus Android arm64 CPU and openEuler with Ascend NPU support. This wide coverage makes b9776 one of the most portable versions of llama.cpp to date.
- Fixes Vulkan Flash Attention overflow by applying bias before softmax (PR #24909)
- Builds for 25+ platform/configurations including macOS, Linux, Windows, Android, iOS, and openEuler
- 118k GitHub stars, 19.9k forks – one of the most popular open-source AI inference projects
Why It Matters
Stable LLM inference on Vulkan GPUs is now more reliable – critical for developers using AMD/Radeon hardware.