Llama.cpp b10150 improves model offloading and fixes DSv4 graphs
New release optimizes GPU offloading logic and enhances DSv4 model support.
The ggml-org team has released llama.cpp version b10150, continuing the development of the high-performance C++ inference engine for LLaMA models. This release focuses on two key areas: improved offloading logic and DSv4 graph fixes. The offloading adjustments (ggml: adjust logic for offloading ops to weight's backend) allow more efficient use of GPU memory by better deciding which operations to offload to the weight's backend, potentially reducing VRAM usage and improving throughput. The DSv4 graph fixes address issues in the graph execution for DeepSeek V4 models, ensuring correct inference on that architecture.
This release provides extensive platform support with pre-built binaries. macOS users get separate builds for Apple Silicon (arm64) and Intel (x64), plus an iOS XCFramework. Linux builds include CPU, Vulkan, ROCm 7.2, OpenVINO, and SYCL variants. Windows builds cover CPU, ARM64, CUDA 12.4 and 13.3, Vulkan, OpenVINO, SYCL, and HIP for AMD GPUs. Android and openEuler builds are also available. The release is signed with a verified GPG key for security. Developers can download from the GitHub releases page or build from source.
- ggml backend: offloading logic adjusted for better GPU memory utilization (#25832)
- llama: DSv4 graph fixes for accurate inference with DeepSeek V4 models
- Pre-built binaries for 30+ platform/backend combinations including CUDA 12/13, ROCm, Vulkan, and SYCL
Why It Matters
Llama.cpp b10150 enables faster, more memory-efficient local LLM inference across desktop, server, and mobile platforms.