Llama.cpp b9637 adds dedicated Cohere2MoE parser for AI models
New release unlocks efficient local inference for Cohere's North Code models.
The open-source llama.cpp project reached version b9637, introducing a dedicated parser for Cohere2MoE (North Code) models. This parser specifically handles the Mixture-of-Experts (MoE) architecture used by Cohere's latest models, allowing users to run them locally with improved efficiency. The commit (aedb2a5) was signed with a verified GPG key, ensuring code integrity.
This release supports an extensive range of hardware backends: macOS on Apple Silicon (arm64) and Intel (x64), iOS via XCFramework, Linux across x64/arm64/s390x with CPU, Vulkan, ROCm 7.2, OpenVINO, SYCL (FP32/FP16) options, Android arm64, Windows (x64/arm64, CPU, CUDA 12/13, Vulkan, SYCL, HIP), and openEuler. Notably, builds with KleidiAI on macOS and certain openEuler configs are disabled in this tag. UI contributions from the community were also merged.
- Dedicated parser for Cohere2MoE (North Code) models in chat interface (#24615)
- Supports 20+ platform configs including macOS, Windows, Linux, Android, iOS
- GPU backends include CUDA 12/13, Vulkan, ROCm 7.2, OpenVINO, SYCL, HIP
Why It Matters
Enables developers to run Cohere's advanced MoE models locally on diverse hardware without cloud dependencies.