Llama.cpp adds Vulkan, OpenVINO and ROCm support
Llama.cpp now runs on Vulkan, OpenVINO and ROCm across Linux, Windows and macOS...
Deep Dive
According to the article, llama.cpp released version b10319. The release contains a fix for `mtmd` to address `longest_edge` ignoring min/max pixels, referencing issue #26638. It also lists the llama.app website and provides download assets for macOS, Linux, Windows, Android, openEuler, and UI components across various configurations.
Key Points
- Llama.cpp v1.0 adds Vulkan, OpenVINO and ROCm GPU backends across Linux, Windows and macOS
- Pre-built binaries target x64/ARM CPUs, AMD/Intel/NVIDIA GPUs and Huawei Ascend accelerators
- Update enables up to multi-GPU and accelerator inference with hardware-specific optimizations
Why It Matters
Expands accessible AI inference to AMD, Intel, NVIDIA and Ascend GPUs, cutting costs and latency for production deployments.