llama.cpp b9844 adds NVFP4 support via ggml-webgpu
New release enables 4-bit floating point inference on GPU with WebGPU.
The llama.cpp project, a popular C/C++ implementation of LLaMA models for local AI inference, released version b9844. The headline feature is the addition of NVFP4 support via the ggml-webgpu backend. NVFP4 is NVIDIA's 4-bit floating point format, which substantially reduces model memory footprint while maintaining accuracy. This integration enables efficient inference on GPUs through WebGPU, a modern graphics API that works across browsers and native apps.
The release comes with extensive platform builds: macOS (Apple Silicon and Intel), Linux (x64/arm64/s390x with CPU, Vulkan, ROCm, OpenVINO, SYCL), Windows (x64/arm64 with CUDA 12/13, Vulkan, OpenCL Adreno, SYCL, HIP), Android (arm64 CPU), and iOS XCFramework. Notably, it includes CUDA 13 DLLs for cutting-edge NVIDIA hardware. This wide compatibility makes advanced quantization accessible to users on virtually any device, democratizing high-performance local AI.
- Adds NVFP4 (NVIDIA FP4) quantization format via ggml-webgpu for GPU inference
- Supports 14+ platform builds including macOS, Windows, Linux, Android, and iOS
- Includes CUDA 12/13, Vulkan, ROCm 7.2, OpenVINO, SYCL, and HIP backends
Why It Matters
Enables efficient 4-bit FP inference on GPUs, reducing VRAM needs for local AI models.