llama.cpp release b10059 optimizes hadamard multiplication on CPU
New default CPU routine for hadamard mul_mat boosts inference efficiency...
The open-source llama.cpp project, a high-performance inference engine for large language models, has released version b10059. This update primarily focuses on a backend change to the hadamard (element-wise) matrix multiplication routine. The default implementation now uses a CPU-based routine instead of previous GPU-accelerated paths. The commit, signed by Aaron Teo from IBM, aims to improve stability and compatibility across diverse hardware configurations.
This release comes with binaries for a wide range of platforms: macOS (both Apple Silicon and Intel, with a separate KleidiAI-enabled variant), Linux (x64, arm64, s390x, plus Vulkan, ROCm 7.2, OpenVINO, and SYCL builds), Windows (CPU, arm64, CUDA 12/13, Vulkan, OpenVINO, SYCL, HIP), Android (arm64 CPU), and even openEuler for specialized NPU hardware. The sheer breadth of supported targets underscores llama.cpp's role as a universal LLM inference tool, allowing users to run models like Llama, Mistral, and others locally without proprietary cloud dependencies.
- Default hadamard mul_mat operation switched to CPU routine for broader compatibility
- Contributed by Aaron Teo (IBM) with GPG-signed commit for verification
- Binaries available for macOS, Linux, Windows, Android, and openEuler across various CPU/GPU backends
Why It Matters
This optimization improves reliability for local LLM inference on CPU, making llama.cpp more accessible on diverse hardware.