llama.cpp b10152 fixes GPU layer allocation for MTP blocks
Multi-token prediction now stays GPU-accelerated with proper layer counting.
llama.cpp, the popular open-source framework for running large language models locally, has shipped version b10152. This release addresses a specific bug in GPU offloading for multi-token prediction (MTP) — a technique that predicts several tokens at once to accelerate generation. Previously, when specifying n_gpu_layers, the allocation didn't correctly count the MTP blocks, causing some front layers to be forced onto the CPU, negating performance gains. The fix ensures that nextn (MTP) blocks are properly accounted for in the layer distribution, keeping critical layers GPU-accelerated.
This version also continues llama.cpp's tradition of broad platform support. Pre-built binaries are available for Windows (x64 CPU, arm64 CPU, OpenCL Adreno, CUDA 12/13, Vulkan, SYCL, HIP), macOS (Apple Silicon and Intel), iOS, Linux (Ubuntu x64/arm64/s390x with CPU, Vulkan, ROCm 7.2, OpenVINO, SYCL FP32/16), Android (arm64 CPU), and openEuler. The fix is particularly impactful for users running models with MTP (like Llama 3.1 variants) on hybrid CPU/GPU setups, reducing latency and improving throughput.
- Fixes GPU layer allocation for multi-token prediction (MTP) blocks to keep front layers on GPU
- Supports over 20 platform variants including macOS, Windows, Linux, Android, and openEuler
- Pre-built binaries include CUDA 12/13, Vulkan, ROCm, OpenVINO, and SYCL backends
Why It Matters
Makes local LLM inference faster and more efficient for multi-token generation workloads.