New Llama.cpp Update Lets You Flip a Switch for Half-Precision AI on Vulkan and NVIDIA GPUs — Here's How

The llama.cpp project, a popular open-source C++ library for running large language models locally, has released version b9733. This update introduces ggml-webgpu adapter toggles specifically for F16 precision on Vulkan and NVIDIA backends. The change allows users to selectively enable or disable half-precision floating-point support via WebGPU, which can improve performance or compatibility depending on hardware and model requirements. The release comes with over 117,000 GitHub stars and extensive platform support.

Version b9733 provides pre-built binaries across a wide range of systems: macOS on Apple Silicon (arm64) and Intel, iOS as an XCFramework, Linux on x86, ARM, and s390x with various GPU backends (Vulkan, ROCm 7.2, OpenVINO, SYCL FP32/FP16), Android arm64, Windows on x64 and arm64 with CUDA 12/13, Vulkan, OpenVINO, SYCL, and HIP, and openEuler with ARM/x86 and ACL graph support. While the detailed release notes were still loading, the core enhancement focuses on fine-tuning GPU execution for better performance across different accelerator types.