llama.cpp b9564 boosts WebGPU with 2D workgroup ops
New release speeds up LLM inference on AMD/Intel GPUs via WebGPU
Deep Dive
The latest llama.cpp release (b9564) implements 2D workgroups for scale, binary, and unary operations in its WebGPU backend. It provides CI builds for macOS (Apple Silicon arm64, Intel x64), Linux (x64, arm64, s390x), Android arm64, Windows (x64, arm64 with CUDA/Vulkan/SYCL/HIP), and openEuler (x86, aarch64).
Key Points
- Implements 2D workgroups in WebGPU for scale, binary, and unary ops
- Reduces kernel launch overhead, improving LLM inference on AMD/Intel GPUs
- Available across macOS, Linux, Windows, and Android builds
Why It Matters
WebGPU optimization expands local LLM inference beyond NVIDIA GPUs, accelerating adoption of on-device AI.