llama.cpp b10004 adds f16 SET_ROWS for Vulkan and CPU backends
New release improves half-precision support across GPU and CPU, fixing Intel issues.
The latest release of llama.cpp, tagged b10004 by the ggml-org team, brings significant improvements to half-precision (f16) operations. The primary change adds full support for f16 as the source type in SET_ROWS operations for both the Vulkan and CPU backends, matching existing f32 capabilities. This enhancement allows more efficient memory usage and faster matrix row manipulation when working with quantized models that use half-precision weights.
Additionally, the release enables DenormPreserve 16 when supported, specifically to address failures observed on Intel GPUs and integrated graphics. The team also expanded the backend test suite to cover these new f16 operations. Builds are available for macOS (Apple Silicon and Intel, with KleidiAI optimizations), iOS, Linux (x64, arm64, s390x with various backends like Vulkan, ROCm, OpenVINO, SYCL), Windows (x64, arm64, CUDA 12/13, OpenCL, Vulkan, OpenVINO, SYCL, HIP), Android (arm64), and others. This release continues to solidify llama.cpp as a versatile and performant framework for running large language models locally on consumer hardware.
- Full f16 support for SET_ROWS in Vulkan and CPU backends, matching f32 performance.
- DenormPreserve 16 enabled to fix accuracy issues on Intel GPUs and integrated graphics.
- Expanded automated testing for f16 operations across all supported backends and platforms.
Why It Matters
Local LLM inference gets faster and more stable on Intel hardware, expanding accessibility of open-source AI.