llama.cpp v10175 adds RDNA3.5 support for AMD GPUs
New release tunes matrix multiplication for AMD RDNA3 and RDNA3.5
llama.cpp, the popular open-source C/C++ implementation for running large language models locally on consumer hardware, has released version b10175. The key change is the addition of RDNA3.5 and RDNA3 to the MMQ (mixed-matrix quantization) configuration files so they can be tuned independently. Previously, AMD GPU tuning was likely combined; now users with RDNA3 or RDNA3.5 GPUs (like AMD Radeon RX 7000 series) can fine-tune matrix multiplication for their specific architecture, potentially improving performance for local LLM inference.
This release includes a wide array of pre-built binaries: macOS Apple Silicon (both standard and KleidiAI-enabled), macOS Intel, iOS XCFramework, Linux for x64 and arm64 with CPU, Vulkan, ROCm 7.2, OpenVINO, SYCL, Android arm64, and Windows variants for CPU, CUDA 12.4/13.3, Vulkan, OpenCL Adreno, and more. The project, led by Georgi Gerganov, continues to be the go-to solution for running models like Llama, Mistral, and Gemma on local hardware.
- Adds separate MMQ config tuning for AMD RDNA3 and RDNA3.5 architectures
- Provides pre-built binaries for macOS, Linux, Windows, Android, and more
- Enables optimized local LLM inference on latest AMD GPUs
Why It Matters
AMD GPU users can now squeeze more performance from local LLMs with architecture-specific tuning in the leading local inference engine.