llama.cpp b9851 fixes CUDA overflow in flash attention kernel
Critical fix for integer truncation and overflow in KQ mask strides
The latest release of llama.cpp, tagged b9851, addresses a critical bug in its CUDA-based flash attention implementation. The fix targets integer truncation and overflow errors that could occur when using KQ mask strides in the flash_attn_mask_to_KV_max kernel. This kernel is responsible for computing key-value maximums during masked attention, a core operation for transformer inference.
While the technical scope is narrow, the fix is significant for users running LLMs on NVIDIA GPUs with CUDA 12 or 13. The release also demonstrates llama.cpp's continued cross-platform support, providing prebuilt binaries for macOS (Apple Silicon and Intel), Linux (x64, arm64, s390x), Windows (CPU, CUDA 12/13, Vulkan, OpenCL), and Android arm64. Users running recent builds on CUDA should upgrade to avoid silent corruption or crashes during attention calculations.
- Fixes integer truncation and overflow in CUDA flash_attn_mask_to_KV_max kernel (PR #24945)
- Commit signed with GitHub verified signature (GPG key B5690EEEBB952194)
- Prebuilt binaries available for 20+ platform combinations including Apple Silicon, Windows CUDA 12/13, and Android arm64
Why It Matters
Stability fix for GPU-accelerated LLM inference with llama.cpp, preventing silent errors during attention computation.