Llama.cpp updates with new hardware support across platforms
Llama.cpp adds CUDA 12.4/13.3, Vulkan, and SYCL support for 20+ platforms
The llama.cpp team has pushed a significant update to their open-source inference framework, vastly expanding hardware compatibility for running Llama models. The new release adds official support for CUDA 12.4 and 13.3 on Windows systems, which will particularly benefit NVIDIA GPU users seeking the latest performance optimizations. Vulkan support has been extended across multiple platforms including Linux, Android, and Windows, providing a cross-vendor graphics API alternative to CUDA.
Additionally, SYCL support has been introduced for Intel GPUs, enabling efficient execution on Intel integrated and discrete graphics cards. The update covers over 20 platform configurations in total, including Apple Silicon (both native and with KleidiAI acceleration), AMD ROCm 7.2 on Linux, and various Intel architectures through OpenVINO and SYCL. This broad hardware support makes Llama.cpp one of the most versatile inference engines available for running large language models on consumer and enterprise hardware alike.
- Added CUDA 12.4/13.3 support for Windows (including DLLs) - 2 new major NVIDIA GPU backends
- Introduced Vulkan support across Linux/Android/Windows and SYCL for Intel GPUs
- Expanded platform coverage to 20+ configurations including Apple Silicon, AMD ROCm, and Intel architectures
Why It Matters
Enables running Llama models on virtually any modern hardware, democratizing LLM access across diverse computing environments.