Llama.cpp updates with new cross-platform support for Llama 3
Llama.cpp rolls out v12.4 with Vulkan, ROCm, and Android ARM support for Llama 3 models
The maintainers of llama.cpp, the open-source C/C++ inference engine for running LLMs locally, have pushed version 12.4 with significantly expanded hardware support. This release adds Vulkan compute support for Ubuntu and Windows, enabling GPU acceleration on a wider range of hardware including integrated graphics chips. For AMD users, ROCm 7.2 support has been added for Radeon GPUs, while NVIDIA users gain CUDA 13.3 compatibility.
The update also introduces Android ARM64 CPU support, allowing developers to run Llama 3 models directly on mobile devices without cloud dependencies. The build matrix now covers 24 different platform configurations including Apple Silicon (both with and without KleidiAI acceleration), Intel x64 CPUs, various ARM architectures, and even niche platforms like IBM s390x mainframes. The team also fixed special token validation issues in this release.
- Llama.cpp v12.4 adds Vulkan GPU acceleration for Ubuntu/Windows and ROCm 7.2 for AMD GPUs
- New Android ARM64 CPU support enables running Llama 3 models directly on mobile devices
- Build matrix expanded to 24 platform configurations including Apple Silicon, Intel, ARM, and IBM s390x
Why It Matters
Expands local LLM deployment options across more hardware, reducing cloud dependency and enabling mobile AI use cases