llama.cpp b10154 adds device listing with common_print_available_devices()
llama.cpp's latest release lets you quickly see all compatible hardware devices.
llama.cpp, the popular open-source C++ library for running large language models locally, has released version b10154. The highlight is a new `common_print_available_devices()` function that enumerates all supported compute backends on the host machine. This is a quality-of-life improvement for developers who need to quickly determine which hardware (e.g., CPU, NVIDIA CUDA, AMD ROCm, Intel OpenVINO, or Vulkan) is available for inference without digging into configuration files.
The release includes an extensive build matrix covering 20+ platform combinations. macOS users get builds for Apple Silicon (with optional KleidiAI optimization) and Intel x64. Linux supports x64 and arm64 with CPU, Vulkan, ROCm 7.2, OpenVINO, and SYCL (FP32 & FP16). Windows gets CPU, arm64, CUDA 12 & 13, Vulkan, OpenVINO, SYCL, and HIP. Android arm64, openEuler (x86 & aarch64 with ACL Graph), and even iOS XCFramework are included. This breadth reflects the project's goal to make local LLM deployment accessible across diverse hardware ecosystems.
- New function `common_print_available_devices()` lists all compatible hardware backends (CPU, GPU, accelerators).
- 20+ build variants across macOS, Linux, Windows, Android, and openEuler with CUDA 12/13, ROCm 7.2, Vulkan, OpenVINO, SYCL, and HIP.
- llama.cp has 122k stars on GitHub and continues to expand platform coverage for local LLM inference.
Why It Matters
Simplifies local LLM deployment by exposing available hardware, crucial for developers optimizing inference on diverse setups.