llama.cpp b10418 adds SYCL host pinned memory for faster GPU transfers
New llama.cpp release optimizes host-to-device memory access on Intel SYCL accelerators.
ggml-org's llama.cpp, the wildly popular C++ library for running large language models locally, released version b10418 on August 13. The headline change is new SYCL support for host pinned memory, which enables faster and more efficient Host-to-Device memory transfers. This directly addresses a critical bottleneck when running LLM inference on Intel GPUs or other SYCL-compatible accelerators. The release also includes a fix for a thread-safety issue in the SYCL backend, making it more reliable for concurrent workloads.
The update maintains llama.cpp's signature breadth of platform support, with prebuilt binaries for Apple Silicon (both standard and KleidiAI-enabled), Linux on x64/arm64/s390x with multiple backends (Vulkan, ROCm 7.14, OpenVINO, SYCL FP16/FP32), Android arm64, and Windows x64/arm64 with CUDA 12/13, Vulkan, OpenVINO, and more. With 124k GitHub stars and 21.7k forks, llama.cpp remains the go-to choice for developers deploying LLMs on edge devices and heterogeneous hardware. This release particularly benefits those using Intel's SYCL programming model to accelerate models on CPUs and GPUs, promising lower latency and better throughput.
- Adds SYCL host pinned memory support to improve Host-to-Device memory access speeds
- Fixes a thread-safety issue in the SYCL backend for reliable concurrent inference
- Ships prebuilt binaries across Windows, Linux, macOS, Android, and openEuler with multiple GPU backends
Why It Matters
Faster memory transfers on SYCL make local LLM inference on Intel hardware more practical for production and edge deployments.