llama.cpp b9537 fixes off-by-one GPU layer bug
Latest llama.cpp release patches a critical off-by-one in GPU layer offloading.
Deep Dive
llama.cpp released b9537 with a fix for off-by-one comparisons to `n_gpu_layers` (PR #24208). The release includes builds for macOS (Apple Silicon and Intel), iOS, Linux (Ubuntu x64/arm64/s390x with various backends), Android (arm64), Windows (x64/arm64 with CUDA 12/13, Vulkan, HIP), openEuler (x86 and aarch64), and UI assets.
Key Points
- Fixes off-by-one comparison in n_gpu_layers handling (issue #24208), improving multi-GPU layer offloading accuracy.
- Pre-built binaries available for 20+ platform/backend combinations, including CUDA 12/13, ROCm, Vulkan, and Apple Metal.
- Project now has 115k stars and 19.2k forks, reflecting its dominance in local LLM inference.
Why It Matters
Essential stability fix for developers running large models across multiple GPUs, preventing crashes and ensuring efficient resource use.