llama.cpp v. b8973 adds CUDA fusion, expands platform support
New release refactors CUDA fusion code and adds builds for macOS, Linux, Windows, Android...
Deep Dive
The llama.cpp project released version b8973, which includes a refactor of the CUDA fusion code. The release provides builds for macOS (Apple Silicon and Intel), Linux (x64 and ARM64), Windows (CPU, CUDA 12 & 13, Vulkan), and Android ARM64.
Key Points
- Major refactor of ggml-cuda fusion code for improved NVIDIA GPU performance.
- Expanded platform support: macOS (Apple Silicon & Intel), Linux (x64/ARM64), Windows (CPU, CUDA 12/13, Vulkan, SYCL, HIP), Android (ARM64).
- New macOS Apple Silicon build with KleidiAI acceleration enabled.
Why It Matters
llama.cpp's latest update broadens hardware compatibility and optimizes CUDA performance, making local LLM inference faster and more accessible.