b8804

The open-source project llama.cpp, maintained by ggml-org, has released a significant update with version b8804. The most notable technical change is the modification to CUDA behavior: the software now requires explicit user opt-in for enabling peer-to-peer (P2P) memory access between GPUs. This change, implemented via pull request #21910, shifts the default from a permissive to a restrictive stance. The primary benefit is enhanced stability and security in multi-GPU environments, as P2P access can sometimes cause driver conflicts or be exploited if not properly managed. By making it an explicit choice, developers have finer control over their system's resource sharing.

The release is packaged with an extensive array of pre-compiled binaries, dramatically simplifying deployment across diverse hardware ecosystems. For Apple users, it provides builds for macOS on both Apple Silicon (arm64) and Intel (x64) architectures, including a special KleidiAI-enabled variant for Apple Silicon. Linux support spans standard CPU builds for x64, arm64, and s390x, plus accelerated versions for Vulkan and ROCm 7.2. Windows users get builds for CPU, CUDA 12.4, CUDA 13.1, Vulkan, SYCL, and HIP. Notably, it also includes specialized builds for Huawei's openEuler OS, targeting their Ascend 310P and 910B AI accelerators. This comprehensive packaging effort makes cutting-edge, efficient local AI inference accessible on nearly any hardware stack, from data center GPUs to edge devices and specialized AI chips.