llama.cpp b9603 adds OpenCL q5_0/q5_1 kernels for Adreno GPUs
Run quantized LLMs faster on mobile GPUs with new OpenCL kernels.
llama.cpp, the leading open-source C/C++ LLM inference engine, just dropped version b9603 with a significant performance boost for mobile and edge devices. The headline addition is full OpenCL support for q5_0 and q5_1 quantization kernels (both GEMM and GEMV) on Adreno GPUs. These kernels are specifically optimized for Qualcomm's mobile GPU architecture, enabling 5-bit quantized models to run efficiently on smartphones, tablets, and other Adreno-powered hardware. The change, contributed by Li He from Qualcomm, marks a major step in bringing high-performance LLM inference to the mobile ecosystem.
Alongside the Adreno OpenCL work, b9603 ships builds for virtually every platform: macOS (Apple Silicon, Intel, KleidiAI), Linux (x64, arm64, s390x, Vulkan, ROCm, OpenVINO, SYCL), Windows (CPU, CUDA 12/13, Vulkan, HIP), Android (arm64 CPU), and even openEuler. The release also includes updated UI assets. This comprehensive platform coverage reinforces llama.cpp's role as the go-to inference engine for developers running LLMs across diverse hardware, from server GPUs to mobile SoCs.
- Adds OpenCL GEMM and GEMV kernels for q5_0 and q5_1 quantization on Adreno GPUs
- Contributed by Qualcomm engineer Li He, optimized for mobile GPU architectures
- Includes builds for macOS, Linux, Windows, Android, and openEuler across CPU, Vulkan, CUDA, ROCm, and more
Why It Matters
Brings efficient 5-bit LLM inference to billions of Qualcomm-powered mobile and edge devices.