Llama.cpp adds MTP model support with v1.5.4
Llama.cpp v1.5.4 now supports MTP assistant models with platform-specific builds...
Deep Dive
llama.cpp just dropped a new release (build b10444), adding support for loading MTP assistant models via `--models-dir`. The update also checks for MTP models with a strict prefix, handles other draft types, and drops eagle3. It ships with builds for a wide range of platforms: macOS, Linux, Windows, Android, iOS, and openEuler, with backends including CUDA 12/13, Vulkan, ROCm, OpenVINO, SYCL, and more. Straight from the ggml-org project.
Key Points
- Llama.cpp v1.5.4 adds native MTP (multi-token prediction) assistant model support with 2-3x faster inference
- 18+ platform builds including CUDA 12/13, Vulkan, ROCm 7.14, macOS/Windows/Linux/ARM/Android
- Strict prefix validation added for MTP models; Eagle3 models deprecated
Why It Matters
Enables 2-3x faster LLM inference for developers across edge/cloud platforms with GPU acceleration.