b8746

The llama.cpp project, a cornerstone of the local AI ecosystem, has rolled out a significant infrastructure update with commit b8746. While the commit itself is primarily a release of pre-compiled binaries, its importance lies in the dramatically expanded hardware support. The update marks the `--split-mode tensor` feature—a method for splitting model layers across different hardware—as experimental, signaling it's for advanced testing.

The key deliverable is the new suite of pre-built binaries, which now includes support for Vulkan graphics APIs on both Linux and Windows, CUDA 13.1 libraries for Windows users, Intel's OpenVINO toolkit for acceleration on Linux, and AMD's ROCm 7.2 platform. This eliminates a major barrier to entry, allowing users to download and run optimized versions of models like Meta's Llama 3 or Mistral's offerings on their specific GPU or CPU without needing to compile the complex C++ codebase themselves. The release also maintains support for Apple Silicon, standard CUDA, and various CPU backends, solidifying llama.cpp's role as the universal runtime for local LLM inference.