llama.cpp b10208 doubles Intel GPU prompt processing with oneMKL flash attention
1473 t/s on Gemma-4-26B — a 1.97x speedup over the TILE path.
ggml-org has shipped llama.cpp b10208, a significant SYCL backend update that brings oneMKL GEMM flash attention to Intel GPUs. The new kernel routes multi-token prefill through XMX-accelerated matrix operations, replacing the slower TILE path. In testing on B70/Battlemage hardware with 32K context and q8_0 KV cache, Gemma-4-26B jumped from 746 t/s to 1473 t/s (1.97x), while Qwen3.6-27B improved from 330 t/s to 609 t/s (1.85x). The release also expands support to all KV cache types — F16, BF16, F32, and quantized — since the kernel converts non-F16 data to F16 before GEMM, eliminating a previous restriction.
The update fixes a critical flash-attention layout bug where the kernel wrote head outputs using a dense head-major layout instead of the interleaved layout expected by TILE's combine function. This corrupted attention for all models except Qwen3.6-27B, whose sparse GQA heads avoided visible overlap. The code also removes redundant SYCL stream waits, adds a GGML_SYCL_ENABLE_MKL_FA environment variable for A/B testing, and gates the MKL path behind a Q->ne[1] >= 32 threshold to keep token generation on the fused VEC kernel. Debug and diagnostic flags (GGML_SYCL_MKL_FA_DEBUG, GGML_SYCL_MKL_FA_DIAG) are included for profiling.
- 1.97x prefill speedup on Gemma-4-26B (1473 t/s vs 746 t/s) and 1.85x on Qwen3.6-27B via oneMKL GEMM flash attention
- Fixes a head-layout corruption bug that silently broke attention for all models except Qwen3.6-27B
- Adds GGML_SYCL_ENABLE_MKL_FA toggle, supports F16/BF16/F32/quantized KV caches, and requires Q->ne[1] >= 32
Why It Matters
Intel GPU users running LLMs locally get nearly 2x faster prompt prefill, making large-model inference feel dramatically more responsive.