Developer Tools

llama.cpp v9534 adds Vulkan FWHT for Intel GPUs with shmem reduction

New release improves AI inference on Intel hardware with shared memory optimization.

Deep Dive

The ggml-org/llama.cpp project released version b9534, introducing Vulkan-based FWHT (fast Walsh-Hadamard transform) support for Intel GPUs with shared memory reduction. This optimization improves performance on Intel integrated and discrete graphics by reducing memory contention. The release also includes workarounds for known driver issues: subgroup shuffle is disabled on MoltenVK (Apple's Vulkan translation layer for AMD GPUs) and AMD, and the FWHT shader is disabled entirely on Intel Windows due to a driver bug. The update is part of ongoing efforts to make LLM inference accessible across diverse hardware, from CPUs to GPUs and accelerators.

This release supports a wide range of platforms: macOS (Apple Silicon Intel, KleidiAI), Linux (x64, arm64, s390x with Vulkan, ROCm, OpenVINO, SYCL), Windows (CPU, CUDA 12/13, Vulkan, HIP), Android (arm64 CPU), and openEuler builds. The inclusion of FWHT with shared memory reduction specifically benefits Intel GPU users, who previously lacked optimized transforms for attention mechanisms. With 115k stars and 19.2k forks, llama.cpp remains a critical tool for running LLMs on edge devices and consumer hardware, and this update closes a performance gap for Intel GPU owners.

Key Points
  • Adds FWHT support for Intel GPUs via Vulkan with shared memory reduction
  • Disables subgroup shuffle on MoltenVK/AMD and FWHT shader on Intel Windows due to driver bugs
  • Supports macOS, Linux, Windows, Android, and openEuler across CPU, CUDA, Vulkan, ROCm, and more

Why It Matters

Improves LLM inference on Intel GPUs, expanding hardware options for local AI deployment.

📬 Get the top 10 AI stories daily