llama.cpp b10456 boosts Intel GPU inference 7.8x with SYCL fix

ggml-org, the team behind the widely-used open-source LLM inference engine llama.cpp, has shipped release b10456 with a targeted performance fix for Intel GPUs. The patch addresses an issue in the quantized cpy kernel launches under the SYCL backend, specifically adjusting thread and block counts to be proportional to the size of the quant. This resolves under- and over-subscription problems that were throttling data transfer throughput. In benchmarks on an Intel Arc 70, the q4_0-to-f32 path saw throughput soar from 20.21 GB/s to 158.19 GB/s—a 7.8x improvement—while other quantization types saw flat but stable performance. The fix is part of ongoing optimization work on llama.cpp, which supports a broad range of hardware backends including CPU, CUDA, Vulkan, ROCm, and SYCL.

For developers deploying local LLMs, this means faster model loading and inference on Intel discrete GPUs, particularly when converting quantized weights to FP32 for computation. It lowers latency and improves the practicality of running models like Llama or Mistral on Intel hardware. The release is available immediately with prebuilt binaries for macOS, Linux, Windows, Android, and more. As llama.cpp continues to mature, such backend-specific optimizations are critical for squeezing maximum performance out of diverse hardware. This update is especially relevant for AI engineers who rely on local inference for privacy-sensitive or cost-sensitive applications, and it signals Intel's growing presence in the AI inference landscape.