b8811
The latest update delivers up to 2x faster inference by eliminating profiling overhead across 27 platform builds.
The ggml-org team behind the massively popular llama.cpp project has released version b8811, a significant performance-focused update that optimizes GPU inference across multiple platforms. The core improvement is the implementation of compute pass batching, which groups GPU operations to reduce overhead and dramatically speed up processing. The team also removed profiling overhead from standard execution paths and fixed critical issues with register tiling matmul implementations, particularly addressing Chrome compatibility problems they attribute to the Dawn WebGPU backend.
This release delivers pre-compiled binaries for an extensive range of 27 hardware and OS combinations, making high-performance local AI more accessible. Builds now cover macOS (both Apple Silicon with new KleidiAI acceleration and Intel), various Linux configurations (CPU, Vulkan, ROCm 7.2, OpenVINO), Windows (with CUDA 12.4, CUDA 13.1, Vulkan, SYCL, and HIP support), iOS via XCFramework, and specialized builds for Huawei's openEuler OS with Ascend 310P and 910B AI processors. The multi-platform approach ensures developers can deploy optimized models from mobile devices to data center hardware.
- Compute pass batching implementation reduces GPU overhead for 2x faster inference on supported hardware
- Fixed register tiling matmul for Chrome/Dawn WebGPU backend and added f32 accumulation improvements
- 27 pre-built binaries covering macOS, Linux, Windows, iOS, and openEuler with specialized AI hardware support
Why It Matters
Enables faster, more efficient local AI deployment across diverse hardware, reducing cloud dependency for LLM applications.