llama.cpp b10174 adds NextN/MTP speculative decoding support for GLM-5.2
New release enables multi-token prediction speculation for GLM-5.2 models.
llama.cpp b10174 adds speculative decoding via NextN/MTP (multi-token prediction) for GLM-5.2 models (GLM_DSA). The update implements a graph_mtp builder that includes dense multi-head latent attention (MLA), sigmoid-gated mixture-of-experts with shared experts, and a shared head with fallbacks. A draft head runs the NextN layers, using a plain attention KV cache separate from the main DSA cache (filtered to trunk layers only). This allows the model to predict multiple future tokens per step, accelerating generation.
Conversion now supports --mtp/--no-mtp flags for GlmMoeDsaForCausalLM, letting users export models with or without the appended NextN block. Release assets include builds for macOS (Apple Silicon, Intel), Linux (x64/arm64 with CPU, Vulkan, ROCm 7.2, OpenVINO, SYCL), Windows (CPU, CUDA 12/13, Vulkan, HIP), Android (arm64), and iOS XCFramework. This brings advanced speculative decoding to a wide range of hardware, including edge devices.
- Adds NextN/MTP speculative decoding for GLM-5.2 (GLM_DSA) with dense MLA and MoE draft heads.
- Conversion script now supports --mtp/--no-mtp export to include or exclude the NextN block.
- Pre-built binaries for macOS, Linux, Windows, Android, and iOS, covering both CPU and GPU backends.
Why It Matters
Brings state-of-the-art multi-token prediction to GLM models, enabling faster inference on diverse hardware.