llama.cpp b9771 trims Vulkan shaders, reduces binary size
New release cuts GPU shader variants with ALIGNED spec constant optimization
The open-source llama.cpp project, a leading framework for running large language models locally, released version b9771 (tagged June 23). This patch targets Vulkan GPU acceleration, specifically the matrix multiplication (mul_mm) kernel. Previously, the ALIGNED flag was baked into separate shader variants, causing combinatorial explosion and inflating binary sizes. By promoting ALIGNED to a specialization constant, the engine now compiles fewer variants dynamically, trimming the final binary size for Vulkan builds.
This release also includes the usual cross-platform builds: macOS (Apple Silicon, Intel), Linux (x64, arm64, s390x), Windows (x64, arm64, CUDA 12/13, Vulkan, OpenVINO, SYCL, HIP), Android arm64, and openEuler. The change is invisible to end users but improves memory efficiency and download times. Developers compiling from source will notice smaller artifact sizes, especially when targeting Vulkan on Linux or Windows. The project continues to refine local LLM inference without sacrificing performance.
- mul_mm ALIGNED made a specialization constant, reducing shader variant count
- Smaller compiled binary sizes for Vulkan builds on Linux, Windows, and Android
- Maintenance release with cross-platform support including CUDA 12/13 and ROCm
Why It Matters
Smaller binaries mean faster downloads and lower memory usage for local LLM inference on GPU-equipped machines.