Llama.cpp b9984 adds Q2_K support to SYCL DMMV path
llama.cpp b9984 brings Q2_K quantization to Intel SYCL backend for faster inference
ggml-org has shipped llama.cpp release b9984, which adds Q2_K quantization support to the DMMV (dequantize-multiply-matrix-vector) reorder path for the SYCL backend. SYCL is a C++-based heterogeneous programming standard, primarily used with Intel GPUs and CPUs. This optimization allows the popular local LLM runtime to leverage lower-bit quantization (2-bit K-quant) more efficiently on Intel and compatible hardware, reducing memory bandwidth and improving inference speed.
The release was tagged on July 13 and signed by Todd Malsbary from Intel, indicating close collaboration with Intel's SYCL toolchain team. It ships precompiled binaries for a wide range of platforms: macOS (Apple Silicon with optional KleidiAI acceleration, Intel x86, iOS), Linux (x86/arm64/s390x CPU, Vulkan, ROCm 7.2, OpenVINO, SYCL FP32/FP16), Android arm64, and Windows (x64/arm64 CPU, OpenCL Adreno, CUDA 12/13, Vulkan, OpenVINO, SYCL, HIP). This broad platform support makes b9984 a significant drop for developers running local LLMs on diverse hardware.
- Adds Q2_K quantization support to the SYCL DMMV reorder path, optimizing 2-bit inference on Intel GPUs and CPUs
- Precompiled binaries for macOS, Linux, Windows, Android, and iOS, including CUDA 12/13, ROCm 7.2, and Vulkan backends
- Release signed by Todd Malsbary (Intel), with commit e474bba, ensuring authenticity and vendor engagement
Why It Matters
Enables faster, memory-efficient local LLM inference on Intel SYCL hardware, expanding quantization options for edge deployments.