llama.cpp v9940 enables offline benchmarking for local LLM inference
New release supports offline param initialization for more accurate performance testing.
The popular open-source project llama.cpp, which enables running large language models locally on consumer hardware, shipped version b9940 on July 9. The key change is a new feature in the `llama-bench` tool: `init params.offline`. This allows the benchmarking parameters to be initialized without requiring internet access, making performance tests more reliable and reproducible, especially in air-gapped environments or for automated CI pipelines.
This release also extends platform support significantly. Pre-built binaries are available for macOS (Apple Silicon and Intel), Linux (x64, ARM64, s390x on Ubuntu), Windows (x64 and ARM64 with CUDA 12 & 13 DLLs, Vulkan, SYCL, OpenCL Adreno, and HIP), Android (ARM64), and even openEuler (with ACL Graph acceleration). The commit was signed with GitHub's verified signature and authored by Adrien Gallouët from Hugging Face, indicating continued collaboration between llama.cpp and the open-source AI ecosystem.
- New `llama-bench` offline initialization (PR #25476) removes network dependency for benchmarking
- Supports CUDA 12 & 13, Vulkan, SYCL, ROCm 7.2, OpenVINO, and OpenCL on Windows, with Apple Silicon and Intel on macOS
- Release includes builds for openEuler (x86 & aarch64) with ACL Graph acceleration for 910b/310p
Why It Matters
Enables developers to reliably benchmark local LLM performance across varied hardware without internet access.