llama.cpp b9847 fixes Gemma E4B MTP FlashAttention on CUDA
A critical CUDA bug in Gemma multi-token prediction is resolved.
The open-source llama.cpp project continues its rapid iteration with release b9847, tagged on June 30. The headline fix addresses a CUDA FlashAttention bug affecting Gemma models that use the E4B MTP (multi-token prediction) variant. Specifically, the commit removes an unused template declaration that was causing issues in the attention kernel, likely leading to incorrect outputs or crashes when running Gemma on NVIDIA hardware with CUDA.
This release builds on the project's wide compatibility, offering precompiled binaries for nearly every platform: Ubuntu (x64 and arm64 with CPU, Vulkan, ROCm 7.2, OpenVINO, SYCL), Windows (x64 and arm64 with CUDA 12/13, Vulkan, OpenVINO, SYCL, HIP), macOS (Apple Silicon with and without KleidiAI, Intel x64), iOS XCFramework, Android arm64, and even openEuler for Ascend NPUs. The cross-platform support ensures that AI practitioners running local LLMs on diverse setups can quickly update to a more stable inference experience.
- Fixes CUDA FlashAttention for Gemma E4B multi-token prediction models
- Removes an unused template declaration that caused inference bugs
- Available across 27 build variants including CUDA, Vulkan, ROCm, SYCL, and mobile
Why It Matters
Stable LLM inference on consumer GPUs requires constant patching; this fix ensures reliable Gemma model performance for devs.