llama.cpp b9523 refactors hparams for multi-token prediction models
Version b9523 improves layer handling, ditching n_layer_kv for next-gen LLM support.
llama.cpp, the popular C/C++ implementation of LLaMA-family models, has released version b9523. This maintenance update focuses on internal code hygiene and compatibility with advanced model architectures. Key changes include a refactor of the hparams.n_layer system, consolidation of layer counting by removing the n_layer_kv() function in favor of a unified n_layer_all, and explicit fixes for nextn layer count handling. Also addressed are duplicate switch cases and proper boolean assignment for extra layers like is_swa and is_recr. These improvements lay the groundwork for supporting multi-token prediction (MTP) models, such as those used by DeepSeek-v2 and certain Mistral variants.
The release covers a wide range of platforms. For Apple users: macOS on Apple Silicon (arm64) and Intel (x64), plus iOS XCFramework. Linux builds target Ubuntu x64/arm64 (CPU), with optional Vulkan, ROCm 7.2, OpenVINO, and SYCL FP32 (disabled). Windows supports x64 and arm64 CPU, plus CUDA 12 and 13 (with respective DLLs), Vulkan, and HIP. Android gets an arm64 CPU build. openEuler also sees updates for x86 and aarch64 with ACL Graph. UI assets are included. Developers running local LLMs with custom architectures should update to ensure stability and future-proofing.
- Refactored hparams.n_layer and replaced n_layer_kv() with unified n_layer_all for simpler layer count management.
- Fixed nextn layer count handling and removed duplicate switch cases to prevent inference errors in complex models.
- Supports MTP (multi-token prediction) architectures and improves extra layer flag logic (is_swa, is_recr).
Why It Matters
Essential for developers running cutting-edge local LLMs—b9523 ensures stable support for advanced architectures like multi-token prediction.