llama.cpp b9664 brings SYCL MoE acceleration for Intel GPUs
New release optimizes MoE matrix multiplication with reordered quantized kernels for SYCL.
Deep Dive
The llama.cpp project released version b9664, adding SYCL support for reordered Q4_K, Q5_K, and Q6_K MoE MUL_MAT_ID. Unsupported 3D reorder cases now fall back instead of aborting. The release also ships builds for macOS (Apple Silicon & Intel), Linux, Android, and Windows (CUDA 12/13, Vulkan, HIP, SYCL).
Key Points
- Adds SYCL MoE support for reordered Q4_K, Q5_K, and Q6_K quantized tensors in MUL_MAT_ID
- Falls back gracefully for unsupported 3D reorder cases instead of crashing
- Ships builds for macOS, Linux, Android, and Windows with CUDA, Vulkan, ROCm, and HIP backends
Why It Matters
Enables faster MoE model inference on Intel GPUs, expanding local AI deployment options for professionals.