llama.cpp b10414 adds TQ2_0 ternary support to Metal backend

llama.cpp, the cross-platform C++ library for running large language models locally, released version b10414 on August 13. The headline feature is new support for GGML_TYPE_TQ2_0 (ternary, 2 bits per element) in the Metal backend. This quantization type packs weights into ternary values (e.g., -1, 0, 1) using just 2 bits, nearly halving memory footprint compared to standard 4-bit formats. The implementation was partially assisted by llama.cpp's own DeepSeek-v4-Flash-0731 model, highlighting the project's use of AI in development.

Beyond TQ2_0, the release optimizes the matrix-vector multiplication (mul_mv) kernel by prioritizing float operations over integer ops, precalculating sums, hoisting coefficients out of inner loops, and ensuring contiguous y-loads. These changes improve inference performance across Metal-supported devices, including all Apple Silicon Macs, iPhones, and iPads. The release also updates prebuilt binaries for macOS, Linux, Windows, Android, and multiple GPU backends (Vulkan, CUDA, ROCm, OpenVINO, SYCL), making it easy for developers to adopt. With 124k stars and 21.7k forks, llama.cpp remains the go-to choice for edge inference, and this update strengthens its position for running compressed models on Apple hardware.