llama.cpp b9867 adds spec-draft-p-min for faster speculative decoding
llama.cpp’s latest release speeds up AI inference with smarter draft selection.
Deep Dive
llama.cpp (by ggml-org) released b9867, adding support for `spec-draft-p-min` in DFlash, plus `n_min` and `n_max` guards. The release includes builds for macOS, Linux, Windows, Android, and iOS, with GPU backends like CUDA, Vulkan, and ROCm.
Key Points
- llama.cpp b9867 adds spec-draft-p-min for controlling draft token probability threshold in DFlash
- Includes n_min and n_max guards to filter low-confidence drafts, improving inference speed by up to 30%
- Available on macOS, Linux, Windows, Android, iOS with GPU backends (CUDA, Vulkan, ROCm, OpenVINO, SYCL, HIP)
Why It Matters
Local AI inference gets smarter and faster, enabling smooth operation on consumer hardware with fewer wasted tokens.