llama.cpp b10322 speeds up SYCL SSM_CONV 1.87x on Intel Arc
Coalesced memory loads cut SSM_CONV latency nearly in half on Arc Pro B70.
llama.cpp's latest tagged release, b10322, delivers targeted performance gains for SYCL-based hardware, particularly Intel discrete GPUs. The core change coalesces the ssm_conv window loads, reducing memory transaction overhead in the SSM_CONV operator used by state-space models like Mamba. On an Arc Pro B70, the optimization achieved a 1.85x speedup (97.68 → 52.95 µs/run) for a 515×3328×1×1 tensor and 1.87x (516.16 → 276.13 µs/run) for a larger 937×8192×1×1 workload, demonstrating significant scaling with problem size.
In practical llama-bench tests on a Qwen3.5 27B Q4_K model (48 of 64 blocks using SSM_CONV), prompt processing improved +2.2% at batch 2048 and +1.8% at batch 512, while token generation stayed flat. The release also highlights new build options, including ROCm 7.2, OpenVINO, SYCL FP16, and Android OpenCL Adreno support, reflecting llama.cpp's broad multi-vendor strategy. Developers running local LLMs on Intel Arc GPUs or SYCL-compatible accelerators should see immediate throughput gains for SSM-based architectures without code changes.
- SYCL SSM_CONV operations run 1.85–1.87x faster on Intel Arc Pro B70 with coalesced window loads
- Prompt processing on Qwen3.5 27B Q4_K improves up to +2.2% at batch 2048 and +1.8% at batch 512
- Adds build targets for ROCm 7.2, SYCL FP16, and Android arm64 CPU alongside existing CUDA, Vulkan, and OpenVINO backends
Why It Matters
Local inference on Intel ARC GPUs gets a free speed boost for Mamba-style models, improving prompt latency without hardware upgrades.