llama.cpp b10048 fixes tensor parallelism for Phi3, Bert, and more
New release patches multi-GPU inference for four model families
The ggml-org team has shipped llama.cpp b10048, a maintenance release that addresses critical tensor parallelism (TP) issues in four model families: Phi3, Bert, Plamo2/3, and ChatGLM. Tensor parallelism is a technique that splits model layers across multiple GPUs to speed up inference, but bugs in previous versions caused errors or crashes for these models. This fix ensures they run reliably in multi-GPU setups, a key requirement for production deployments.
The release builds span all major platforms: macOS (Apple Silicon and Intel), Linux (x64, ARM64, s390x) with CUDA, Vulkan, ROCm, and OpenVINO support, Windows (x64, ARM64) with CUDA 12/13, Vulkan, and HIP, plus Android, iOS, and custom builds for openEuler. Users can download precompiled binaries or rebuild from source using the tagged commit 2e1fd76. The update is particularly relevant for developers running local LLM inference on multi-GPU workstations or cloud instances.
- Fixes tensor parallelism for Phi3, Bert, Plamo2/3, and ChatGLM models
- Release includes builds for macOS, Linux, Windows, and Android with GPU backends like CUDA 12/13, Vulkan, and ROCm
- Patch addresses multi-GPU stability issues that previously caused errors during inference
Why It Matters
Smooth multi-GPU inference for these models unlocks faster, more reliable local LLM deployments.