llama.cpp b10094 auto-detects speculative decoding sidecars
No more manual spec-type flags with draft repo sidecars – it just works.
The latest release of llama.cpp (b10094) from ggml-org brings a smart quality-of-life improvement for users running speculative decoding with llama-server. When pointing the `-hfd` flag at a Hugging Face repo that contains sidecar files (mtp, dflash, or eagle3), the software now automatically infers the correct speculative type, removing the need for the `--spec-type` argument. Previously, without an explicit type, the draft resolved to the full model, defeating the purpose of speculative decoding.
Under the hood, the inference follows a priority order: mtp > dflash > eagle3. This means the first available sidecar type is chosen automatically. An explicit `--spec-type` flag still takes precedence, and repos without sidecars behave as before (full model as draft). The release is available for macOS (Apple Silicon, Intel), Linux (x64, arm64, Vulkan, ROCm, OpenVINO), Windows (x64, arm64, CUDA, Vulkan, OpenCL), Android, and iOS. This is a targeted but impactful fix that simplifies local LLM experimentation and deployment for developers working with speculative decoding.
- llama.cpp b10094 auto-detects speculative decoding type from draft repo sidecar files (mtp, dflash, eagle3) without needing `--spec-type`
- Sidecar detection follows priority: mtp > dflash > eagle3, picking the first available type found in the repo
- Works seamlessly with `llama-server -hf repo:Q3_K_M -hfd repo:Q8_0` – overridable by an explicit `--spec-type` flag
Why It Matters
Eliminates a common manual step, making speculative decoding easier for local LLM inference.