llama.cpp b9993 adds Hy3 (Hunyuan 3) with MTP speculative decoding

The llama.cpp project, a popular C/C++ inference engine for large language models, has rolled out version b9993. The headline feature is support for Tencent's Hunyuan 3 (Hy3) model architecture. Hy3 is a Mixture-of-Experts (MoE) decoder stack with several novel components: per-head Q/K RMSNorm, a sigmoid router that uses expert selection bias, an always-active ungated shared expert, and leading dense blocks (first_k_dense_replace). The implementation also includes MTP (Multi-Token Prediction) speculative decoding, which can accelerate inference by predicting multiple tokens ahead. The source code was adapted from a community fork by charlie12345 and integrated into the mainline llama.cpp API.

This release also packs builds for a wide range of platforms: macOS (Apple Silicon with and without KleidiAI, Intel), Linux (x64 and ARM64 CPU, Vulkan, ROCm 7.2, OpenVINO, SYCL FP32/FP16), Windows (x64 and ARM64 CPU, CUDA 12/13, Vulkan, OpenCL Adreno, HIP), and Android (ARM64 CPU). Developers and AI enthusiasts can now run the Hy3 model locally with optimized performance. The addition of Hy3 expands llama.cpp's already extensive model support, making cutting-edge MoE architectures accessible to the open-source community.