llama.cpp b9934 boosts WebGPU flash attention performance
New subgroup split tuning speeds up LLM inference on GPUs via WebGPU.
Deep Dive
llama.cpp's b9934 release (July 9) tunes the subgroup split (d_split) in flash_attn_vec for ggml-webgpu. Builds are available for macOS, Linux, Windows, Android, supporting CUDA 12/13, Vulkan, ROCm 7.2, OpenVINO, SYCL, and HIP.
Key Points
- Tunes the subgroup split (d_split) in flash_attn_vec for WebGPU to improve attention efficiency.
- Release b9934 includes builds for macOS (Apple Silicon, Intel), Linux, Windows, Android (arm64), and openEuler.
- Supports backends: CUDA 12/13, Vulkan, ROCm 7.2, OpenVINO, SYCL, HIP, and OpenCL.
Why It Matters
Enables faster on-device LLM inference via WebGPU, making browser-based AI more practical.