llama.cpp b9774 boosts Vulkan backend with new ops support
New release adds SQR, SQRT, SIN, COS, CLAMP, LEAKY_RELU, NORM for Vulkan
The ggml-org team has shipped llama.cpp b9774, a significant update to the popular local LLM inference engine. This release focuses heavily on the Vulkan backend, introducing support for all backend tests for SQR, SQRT, SIN, COS, CLAMP, LEAKY_RELU, and NORM operations. These changes streamline GPU compute by consolidating SQR/SQRT/SIN/COS/CLAMP/LEAKY_RELU into a single unary.comp shader, improving maintainability. Additionally, NORM now handles noncontiguous data, with corresponding noncontiguous row test cases added for both norm and l2_norm operations, along with CPU backend and l2_norm.comp fixes. The release also includes updated supports_op checks for CUDA and WebGPU backends.
The release comes with a wide array of prebuilt binaries covering macOS (Apple Silicon and Intel, plus KleidiAI enabled), iOS (XCFramework), Linux (x64, arm64, s390x with Vulkan, ROCm 7.2, OpenVINO, SYCL), Windows (x64, arm64, CUDA 12/13, Vulkan, OpenVINO, SYCL, HIP), Android (arm64 CPU), and even openEuler. While the core change is backend optimization, this update ensures llama.cpp remains performant and portable across diverse hardware, particularly for users leveraging Vulkan-based GPUs.
- Vulkan backend now passes full test suite for 7 new operations: SQR, SQRT, SIN, COS, CLAMP, LEAKY_RELU, NORM
- Unary operations consolidated into a single shader (unary.comp), with NORM and L2_NORM now supporting noncontiguous tensors
- Binary builds available for 25+ platform combinations including Windows CUDA 12/13, macOS KleidiAI, and Android arm64
Why It Matters
llama.cpp's Vulkan improvements expand GPU compatibility for running local LLMs beyond NVIDIA, enabling broader hardware utilization.