llama.cpp b9531 boosts tensor parallel performance with 128 granularity
New release rounds up TP granularity to 128 for smoother GPU scaling.
Deep Dive
llama.cpp b9531 released. Key change: tensor parallelism (TP) granularity now rounds up to 128. Also removed an assert.
Key Points
- TP granularity rounded up to 128 to align with GPU memory alignment requirements
- Pre-built binaries for macOS, Linux, Windows, Android, and openEuler with multiple backends (CUDA, Vulkan, ROCm, etc.)
- Signed with GitHub verified signature (GPG key B5690EEEBB952194) for secure downloads
Why It Matters
A simple alignment fix that improves multi-GPU inference performance for local LLM users with no code changes.