llama.cpp b9802 ships cross-platform support with CUDA 12/13 and Vulkan

ggml-org has tagged and released llama.cpp b9802, the newest iteration of its widely-used C/C++ inference engine for running LLaMA-family large language models locally. This release significantly expands hardware compatibility by offering precompiled binaries for a broad range of platforms — from macOS Apple Silicon (ARM64) with and without KleidiAI acceleration, to Intel x64 Macs, and a dedicated iOS XCFramework. On the Linux side, users can choose from CPU-only builds for x64 and ARM64, as well as GPU-accelerated variants using Vulkan, ROCm 7.2, OpenVINO, SYCL (both FP32 and FP16), and even builds for IBM s390x mainframes. Windows users get x64 and ARM64 CPU options, plus a Windows ARM64 build optimized for Adreno OpenCL, and multiple CUDA backends including CUDA 12.4 and CUDA 13.3 DLLs. There are also Vulkan, OpenVINO, and HIP variants for Windows, plus openEuler builds for ARM and x86 with ACL graph support.

This release continues llama.cpp’s mission to democratize local LLM inference by removing OS and GPU vendor lock-in. Developers and AI enthusiasts can now deploy models like Llama, Mistral, and Mixtral on everything from a Raspberry Pi-like ARM64 device to high-end AMD or NVIDIA GPUs. The inclusion of ROCm 7.2 support is particularly notable for AMD GPU users, while the SYCL builds cater to Intel GPU and XPU users. The CUDA 13.3 DLLs suggest forward-looking compatibility with NVIDIA’s latest driver stack. With precompiled binaries available for immediate download, b9802 lowers the barrier for anyone wanting to run state-of-the-art LLMs privately on their own hardware, without cloud dependencies.