llama.cpp b10007 fixes OpenCL dp4a bug for compatibility
New release ensures GPU inference works on devices without cl_khr_integer_dot_product.
The ggml-org/llama.cpp project, the backbone for running large language models on consumer hardware, just shipped version b10007. This release focuses on fixing an OpenCL backend bug that caused failures on devices where the `cl_khr_integer_dot_product` extension is unavailable. Previously, the backend would either crash or misbehave when trying to call dp4a kernels on unsupported hardware. The two main changes: (1) ensure the backend does not fail its initialization if the extension is missing, and (2) avoid calling dp4a kernels when dot product operations are unavailable. This is a targeted fix but important for users running LLMs on diverse GPUs (e.g., older AMD, Intel, or mobile GPUs via OpenCL).
The release package includes build artifacts for macOS (Apple Silicon with and without KleidiAI, Intel), Linux (x64/arm64 CPU, Vulkan, ROCm 7.2, OpenVINO, SYCL FP32/FP16), Android (arm64 CPU), and Windows (x64/arm64 CPU, OpenCL Adreno, CUDA 12/13, Vulkan, OpenVINO, SYCL, HIP). It also covers OpenEuler builds. By fixing this OpenCL compatibility issue, llama.cpp improves stability for a wider range of hardware, reinforcing its role as the go-to inference engine for local AI enthusiasts and professionals who need reliable cross-platform performance.
- Fixes OpenCL dp4a kernel crash on devices without cl_khr_integer_dot_product extension
- Backend initialization no longer fails on unsupported OpenCL devices
- Release includes builds for 7+ platforms including macOS, Linux, Windows, Android, and OpenEuler
Why It Matters
Broadens local LLM inference compatibility, especially for older GPUs with limited OpenCL extensions.