Developer Tools

llama.cpp's b10359 adds WebGPU fixes, i32 support

⚡The 124k-star local LLM runner ships WebGPU CI fixes and broader tensor support.

Deep Dive

ggml-org's llama.cpp, the go-to C++ library for running LLMs locally, released b10359 on August 11. This incremental update focuses on stabilizing the WebGPU backend by fixing CI errors referenced in issues #25025 and #25262. The commit also rebases and refines the new flash attention test, disabling subgroup matrices when max_kv_tile equals zero, and removes stray log output. A notable functional change is added i32 (32-bit integer) support to the cpy operation, which now enables the all-ops test to pass—a step toward broader tensor type compatibility for edge cases.

The release also restores non-target CI tests and comments out a TODO in build-cpu.yml, tightening the project's continuous integration pipeline. llama.cpp continues to support an extensive range of hardware and OS targets, with prebuilt binaries for macOS (Apple Silicon and Intel), iOS, Linux (x64, arm64, s390x, plus Vulkan, ROCm 7.14, OpenVINO, and SYCL variants), Android, Windows (CPU, CUDA 12/13, OpenCL Adreno, Vulkan, and more), and openEuler. With 124k GitHub stars, llama.cpp remains the most popular tool for on-device LLM inference, and b10359 reinforces its stability across diverse deployment scenarios.

Key Points
  • WebGPU CI errors from issues #25025 and #25262 are fixed in commit 84f7129
  • Adds i32 support to the cpy op, enabling the all-ops test suite to pass
  • Binaries available for macOS, Linux, Windows, Android, iOS, and openEuler across CPU, CUDA, Vulkan, ROCm, and OpenVINO backends

Why It Matters

For teams running LLMs on local or edge hardware, this release improves WebGPU reliability and tensor compatibility across platforms.

📬 Get the top 10 AI stories daily