llama.cpp vb9692 adds broader GPU backend support and fixes multimodal model
Run LLMs on nearly any device: Apple Silicon, Android, Windows, Linux, and more.
The ggml-org team has released llama.cpp version b9692, the latest milestone for the highly optimized C++ inference engine that enables running large language models locally on consumer hardware. This update broadens platform compatibility to cover macOS (both Apple Silicon and Intel), iOS, Linux (x64, arm64, s390x), Android arm64, and Windows (x64 and arm64). The pre-built binaries and XCFramework for iOS allow developers to integrate local AI into mobile apps without cloud dependencies.
The release also expands GPU backend choices: Vulkan for cross-platform acceleration, CUDA 12/13 for NVIDIA GPUs, ROCm 7.2 for AMD, OpenVINO for Intel, SYCL for heterogeneous systems, and HIP for portable GPU code. A notable fix addresses the llava_uhd multimodal model, which no longer uses an incorrect batch dimension, improving reliability for vision-language tasks. With 117k stars on GitHub and an active community, llama.cpp continues to democratize local AI by making it accessible on everything from phones to workstations.
- Supports macOS (Apple Silicon & Intel), iOS, Linux (x64, arm64, s390x), Android arm64, and Windows (x64, arm64).
- Backends include CPU, Vulkan, CUDA 12/13, ROCm 7.2, OpenVINO, SYCL (FP32/FP16), and HIP.
- Fixes llava_uhd multimodal model's batch dimension issue (#24732) for better vision-language inference.
Why It Matters
Enables developers to deploy LLMs locally across devices, from phones to servers, with multiple GPU options.