Developer Tools

llama.cpp b10148 fixes draft model override and sidecar resolution

New release ensures explicit --model-draft flags aren't silently overridden

Deep Dive

ggml-org’s llama.cpp b10148 addresses a critical speculative decoding workflow issue. Previously, an explicit --model-draft file provided with the -hfd flag was silently overridden by the sidecar resolution of the draft repository. This meant manual CLI configuration lost precedence, causing confusion and failed inference runs. The fix (PR #26165) now disables sidecar resolution when an explicit draft file is given, ensuring the user’s choice wins.

Additionally, the release refines sidecar selection to anchor on the draft plan’s tag directly: it attempts an exact tag match first, then falls back to the closest quantization available. The sidecar can now resolve even when no full model matches the tag, and a wired draft sidecar counts as explicit, preventing unnecessary downloads. Speculative load logs are promoted from trace to info level, providing clearer visibility into draft model loading and MTP context—bringing consistency with mmproj and primary logs. These changes make speculative decoding more predictable and easier to debug.

Key Points
  • Fixes explicit -md flag being overridden by automatic sidecar resolution (PR #26165)
  • Sidecar selection now anchors on draft plan’s tag with exact match, then closest quant
  • Speculative load logs promoted from trace to info for easier monitoring of draft models

Why It Matters

Reliable draft model handling improves speculative decoding performance and developer debugging in llama.cpp.

📬 Get the top 10 AI stories daily