b8893
The update flips a key switch, unlocking major performance gains for AMD ROCm users running local LLMs.
The open-source project llama.cpp, maintained by ggml-org, has released a significant update with commit b8893. The core technical change is flipping the `GGML_HIP_GRAPHS` configuration to be enabled by default for users running on AMD's ROCm platform. This reverses a decision made in 2023 (pull request #11362) where the feature was disabled due to negative performance impacts at the time. The commit notes that "improvements in rocm and our usage and construction of graphs" have now made HIP graphs a net positive for speed, particularly in reducing kernel launch overhead during AI model inference.
For developers and users, this means pulling the latest llama.cpp code will automatically leverage graph capture and replay on supported AMD GPUs, leading to faster model loading and more efficient execution of models like Llama 3, Mistral, and others. The update is part of a broader release that includes pre-built binaries for a wide range of platforms, including Windows x64 (HIP), Ubuntu x64 (ROCm 7.2), macOS Apple Silicon, and various CPU and Vulkan backends. This change lowers the barrier to high-performance local AI on AMD hardware, making it more competitive with NVIDIA's CUDA ecosystem for mainstream users.
- Commit b8893 enables HIP graphs by default for AMD ROCm users, reversing a prior performance-based disable.
- Leverages improvements in both ROCm software stack and llama.cpp's own graph construction methods for speed gains.
- Part of a full release with binaries for Windows HIP, Ubuntu ROCm 7.2, macOS, iOS, Linux CPU/Vulkan, and Android.
Why It Matters
Delivers significantly faster inference for local LLMs on AMD GPUs, making high-performance AI more accessible and competitive with NVIDIA.