Llama.cpp MTP support now in beta!

Llama.cpp, the open-source C++ inference engine for large language models, has introduced beta support for Multi-Token Prediction (MTP), a technique that predicts multiple tokens at once to significantly boost generation throughput. The feature was contributed by developer Aman, building on earlier issues and discussions from the community. Currently, MTP is only supported for the Qwen3.5 model family, but the announcement hints that support for other architectures will follow soon.

This development, combined with llama.cpp's maturing tensor-parallel support, is expected to erase most performance gaps with vLLM—a competing inference engine popular in production AI deployments. For local AI enthusiasts and professionals running models on consumer hardware, this means faster text generation without sacrificing accuracy or requiring expensive cloud GPUs. As MTP and tensor-parallel features stabilize, llama.cpp could become the default choice for high-performance local inference.