llama.cpp b9129 adds adaptive CPU fallback for small batches

llama.cpp, the widely used C/C++ implementation for running large language models locally, has shipped version b9129. The highlight is a new adaptive fallback feature for the ggml-zendnn backend, which uses Intel's oneDNN library. When batch sizes are small (e.g., single-user chat), GPU acceleration can introduce overhead that actually slows down inference. This release solves that by automatically falling back to the CPU backend for such cases, ensuring optimal latency regardless of batch size.

Users can toggle this behavior via the new environment variable GGML_ZENDNN_ADAPTIVE_FALLBACK (default: enabled). When disabled, the original fallback logic is restored. The release also includes updated binaries for all major platforms: macOS (Apple Silicon and Intel, with KleidiAI option), Linux (x64/arm64 with CPU, Vulkan, ROCm 7.2, OpenVINO, SYCL), Windows (x64/arm64 with CPU, CUDA 12/13, Vulkan, SYCL, HIP), Android, iOS, and openEuler. This makes b9129 a valuable update for developers deploying LLMs in production or running them on edge devices.