llama.cpp b9484 speeds up large batch inference on OpenCL GPUs

The latest release of llama.cpp, tagged b9484, brings a targeted performance optimization for OpenCL-backed GPU inference. Specifically, it adds flat variants of the q4_K and q6_K quantized kernels for gemv (general matrix-vector multiplication) operations when the batch size M is very large. This change is especially beneficial for users running large language model inference on AMD GPUs (via ROCm), Intel GPUs, or any OpenCL-compatible hardware, as it reduces memory overhead and improves computational efficiency.

Beyond the OpenCL improvements, this release continues llama.cpp's tradition of broad platform support. It ships binaries for macOS (Apple Silicon and Intel, with KleidiAI enabled on Arm), Linux (x64, arm64, s390x with CPU, Vulkan, ROCm 7.2, OpenVINO, SYCL FP32), Android (arm64 CPU), and Windows (x64 and arm64 CPU, CUDA 12/13 DLLs, Vulkan, HIP). Some configurations like macOS Intel with KleidiAI and openEuler builds are disabled in this release. The commit was signed with a verified GitHub signature.

For developers and AI engineers who run LLMs locally or in edge deployments, this update means faster inference on a wider range of GPU hardware, particularly when processing large batches of prompts. The flat kernel variants reduce register pressure and improve cache utilization, which can translate to lower latency and higher throughput in production environments. As always, llama.cpp remains focused on efficient inference without requiring proprietary backends.