llama.cpp b9786 adds OpenCL non-contiguous norm support
New release enhances GPU inference with non-contiguous row normalization
The latest llama.cpp release, tagged b9786, delivers a key enhancement for GPU-accelerated LLM inference: OpenCL now supports non-contiguous rows in normalization operations. This technical improvement allows the popular C/C++ LLM inference engine to handle more varied tensor layouts without requiring data copying, reducing memory overhead and improving performance on OpenCL-compatible devices (e.g., AMD GPUs, Intel GPUs, and Adreno mobile GPUs).
The release maintains comprehensive platform coverage, offering prebuilt binaries for 27 different configurations. Notably, Apple Silicon builds are included with and without KleidiAI (ARM performance libraries), though KleidiAI is disabled for this version. Windows users get CUDA 12.4/13.3 DLLs and HIP support for AMD ROCm. Linux targets span x64, arm64, and even s390x mainframes, with Vulkan, ROCm 7.2, OpenVINO, and SYCL backends. Mobile developers benefit from Android arm64 and iOS XCFramework builds. The release also includes UI assets, presumably for the built-in web UI.
- OpenCL normalization now supports non-contiguous rows, reducing memory transfers for irregular tensor shapes
- 27 platform builds shipped: macOS, Linux (x64/arm64/s390x), Windows (x64/arm64), Android, iOS
- GPU backends include CUDA 12/13, ROCm 7.2, Vulkan, SYCL FP16/32, OpenVINO, and OpenCL Adreno
Why It Matters
More flexible GPU inference for llama.cpp users with non-NVIDIA hardware, improving performance on AMD/Intel/Adreno GPUs.