llama.cpp b10344 adds MTP support for NVIDIA Nemotron models

llama.cpp, the popular C++ inference engine for running LLMs locally, has shipped a new release (b10344) that introduces multi-token prediction (MTP) support for NVIDIA's Nemotron model family, including Nemotron Nano. MTP is a decoding technique where the model predicts multiple future tokens simultaneously rather than generating one token at a time. This reduces the number of inference steps needed, cutting latency and improving throughput for text generation on both CPU and GPU backends.

This release also adds new mtp_flags for fine-grained control over MTP behavior, allowing developers to toggle and tune multi-token generation within llama.cpp's API. The project maintains an exceptionally broad platform matrix: the b10344 build includes binaries for Windows x64 with CUDA 12.4 and 13.3 DLLs, Linux with CPU, Vulkan, ROCm 7.2, OpenVINO, SYCL, and s390x, plus Android arm64, macOS Apple Silicon (including a KleidiAI-optimized variant), and more. For users already using llama.cpp for local inference, upgrading to b10344 enables MTP acceleration on Nemotron models without needing to modify existing pipelines — just pull the binary and enable the flag. The release is signed and verified, and the builds are available at llama.app.