llama.cpp b10284 fixes MTP memory allocation bug with 123k stars
llama.cpp's new release patches a memory bug in MTP layers for smoother multi-token prediction.
llama.cpp, the widely used open-source C++ library for running LLaMA and other large language models locally, has released a new tagged version: b10284. Maintained by ggml-org, the project currently boasts over 123k stars and 21.4k forks on GitHub, making it one of the most popular AI inference engines available. The key change in this release is a single, targeted fix: a memory allocation issue for MTP (Multi-Token Prediction) layers, referenced in commit #26605. Multi-token prediction is a technique that lets models generate several tokens at once, speeding up decoding; the bug could cause crashes or instability when using models with MTP layers. This patch corrects the buffer sizing logic, ensuring proper memory assignment.
The release package includes a broad set of prebuilt binaries for various platforms and hardware accelerators. Users can download builds for macOS (Apple Silicon, including a KleidiAI-enabled variant, and Intel x64), iOS, Linux (x64, arm64, s390x CPU; Vulkan, ROCm 7.2, OpenVINO, SYCL FP32/FP16), Android arm64, and Windows (x64/arm64 CPU, CUDA 12.4 or CUDA 13.3, Vulkan, OpenVINO, SYCL, HIP, and OpenCL Adreno). OpenEuler builds for Ascend NPUs are also listed but currently disabled. The project also provides separate UI assets. This release focuses on stability rather than new features, making it a safe upgrade for developers relying on local inference pipelines that use MTP layers.
- llama.cpp b10284 fixes a memory allocation bug in MTP (Multi-Token Prediction) layers.
- The project has 123k GitHub stars and 21.4k forks, reflecting its massive adoption.
- Prebuilt binaries are available for CUDA 12/13, ROCm, Vulkan, OpenVINO, SYCL, and CPU across Windows, Linux, macOS, Android, and iOS.
Why It Matters
For developers running local LLMs, this fix improves stability in multi-token prediction workloads, reducing crashes in high-throughput inference.