llama.cpp b10057 fixes SYCL bugs for Intel GPU inference
Critical bugfix release improves Q5_K kernel handling and row calculations on Intel hardware.
The llama.cpp repository released version b10057 on July 17, 2024, authored by Todd Malsbary from Intel. This patch release focuses on fixing multiple SYCL (Intel's oneAPI DPC++/C++ compiler) backend issues that affected users running LLM inference on Intel GPUs. The most critical fix corrects row calculation when K_QUANTS_PER_ITERATION is set to 1, which previously caused incorrect tensor indexing. Additionally, the team addressed a missing second-half processing step in the reordered Q5_K kernel (a 5-bit quantization type for model weights), ensuring all operations complete properly.
Other improvements include a consistent use of K_QUANTS_PER_ITERATION across all Q5_K kernels, a fix for a potential off-by-one error, and a missing upper bound check on row count. While the changes are primarily for the SYCL backend, they collectively improve stability and performance for users deploying local LLMs on Intel Arc GPUs and integrated graphics. The release is signed with GitHub's verified signature and includes build artifacts for a wide range of platforms, from macOS (Apple Silicon and Intel) to Linux (x64/arm64 with Vulkan, ROCm, OpenVINO, or SYCL), Windows, and Android.
- Fixed row calculation bug in SYCL backend when K_QUANTS_PER_ITERATION=1
- Added missing second-half processing to reordered Q5_K kernel for Intel GPUs
- Consistent use of K_QUANTS_PER_ITERATION across all Q5_K kernels
Why It Matters
Improves local LLM inference stability on Intel GPUs, expanding accessible hardware for open-source AI.