b8926

The ggml-org/llama.cpp project, a popular open-source library for running large language models (LLMs) locally, released version b8926 on April 25. This update introduces WebGPU support for SSM_SCAN, a computational operation used in state-space models (SSMs) like Mamba. SSM_SCAN enables efficient parallel processing of sequential data, which is critical for AI models that handle long sequences. The release also blocks set_rows error checking, reducing potential issues during graph computation. This enhancement is part of ongoing efforts to optimize llama.cpp for diverse hardware, including GPUs via WebGPU, a modern graphics API that runs across platforms (e.g., Chrome, Edge, Safari).

The update comes with a wide range of prebuilt binaries for developers and users. For macOS, it supports Apple Silicon (arm64) and Intel (x64) variants, including a KleidiAI-enabled version for Apple's Neural Engine. Linux users get CPU builds for x64, arm64, and s390x, plus GPU-accelerated builds via Vulkan (x64 and arm64), ROCm 7.2 (AMD GPUs), OpenVINO (Intel), and SYCL (Intel and AMD). Windows builds cover CPU (x64 and arm64), CUDA 12 and 13 (NVIDIA GPUs), Vulkan, SYCL, and HIP (AMD). Android is supported with an arm64 CPU build. This broad compatibility makes llama.cpp a go-to tool for running AI models locally on consumer hardware, with WebGPU integration potentially enabling in-browser AI inference.