b8167
The popular open-source inference engine adds Vulkan, CUDA 13, and ROCm 7.2 support across 23 pre-built binaries.
The ggml-org team has released commit b8167 for llama.cpp, the widely-used open-source inference engine that enables efficient local execution of Meta's Llama models. This release represents a significant expansion in hardware compatibility, providing 23 pre-built binaries across major operating systems and compute backends. The update specifically addresses a token padding issue (mtmd: fix padding of n_tokens #19930) while dramatically broadening deployment options for developers and researchers who need to run large language models on various hardware configurations without extensive compilation.
The technical scope is substantial: macOS support includes both Apple Silicon (arm64) and Intel (x64) variants, while Linux builds now feature Vulkan and ROCm 7.2 support alongside traditional CPU implementations. Windows users gain access to CUDA 12.4 and 13.1 DLLs, Vulkan, SYCL, and HIP backends. The release also includes specialized builds for openEuler with Huawei Ascend 310p and 910b support via ACL Graph. This multi-backend approach allows developers to leverage GPU acceleration across NVIDIA, AMD, Intel, and Apple hardware while maintaining the project's signature efficiency for CPU-only deployment scenarios.
- Adds 23 pre-built binaries across macOS, Linux, Windows, and openEuler with specialized hardware support
- Introduces Vulkan, CUDA 13.1, ROCm 7.2, and SYCL backends alongside existing CPU implementations
- Fixes token padding issue (#19930) in the mtmd component while expanding deployment flexibility
Why It Matters
Dramatically lowers the barrier for local AI deployment by providing optimized binaries for diverse hardware, from consumer GPUs to enterprise accelerators.