b8860

The maintainers of the massively popular open-source project Llama.cpp have pushed a targeted but critical update with release b8860. This version specifically fixes a bug (#22129) in the tensor-parallel implementation that affected Google's Gemma-4 MoE (Mixture of Experts) model. The issue was a 'delayed AllReduce' operation, a core communication step in distributed computing where results from different GPUs are combined. When running Gemma-4 across multiple GPUs, this bug could cause the model to skip or incorrectly process this synchronization, leading to potentially garbled or incorrect text generation. The fix ensures the computational graph correctly traverses nodes and chains of operations (MULs) to perform the AllReduce at the proper time.

The release is notable for its extensive cross-platform support, providing pre-compiled binaries that make deployment straightforward for users. It covers a wide range of systems including macOS on both Apple Silicon and Intel, various Linux distributions (Ubuntu, openEuler) with CPU, Vulkan, and ROCm backends, Windows with CUDA 12/13, Vulkan, and SYCL support, and even Android. This broad compatibility underscores Llama.cpp's role as a universal runtime for running large language models efficiently on diverse hardware, from data center GPUs to consumer laptops and mobile devices. For developers and researchers using Gemma-4 in multi-GPU configurations, applying this update is essential for ensuring the stability and correctness of their inference workloads.