llama.cpp b9878 fixes tensor-split params for draft models and GQA attention
Critical patch for speculative decoding with multi-GPU setups and GQA attention metadata.
Deep Dive
The ggml-org/llama.cpp project released version b9878, fixing stale tensor-split parameters for draft models (issue #24814) and correcting tensor split metadata for GQA attention. The release also reverts a ggml-backend-meta asset hack, and a prior attempt to tidy code style was itself reverted. Available on all major platforms including Apple Silicon, Linux, Windows (CUDA/Vulkan), and Android.
Key Points
- Fixes stale tensor-split parameters for draft models in speculative decoding (#24814)
- Corrects tensor split metadata for Grouped Query Attention (GQA) attention
- Available on macOS, Linux, Windows, Android with multiple GPU backends
Why It Matters
Ensures correct multi-GPU inference for draft models and GQA architectures in local LLM deployments.