llama.cpp b9557: CUDA memory fix and expanded platform support
New release fixes CUDA context reset issue for better GPU memory handling.
The llama.cpp project, led by Georgi Gerganov and community contributors, has released version b9557, a maintenance update focused on stability and broader platform support. The most notable change is a fix for CUDA context reset after reading memory size, addressing a bug that could cause memory allocation errors when switching between GPU and CPU backends. The commit also refines the get_memory function to properly handle device and host buffer counting while excluding HIP and MUSA backends from this logic. A device mutex replaces an atomic operation for safer concurrent access.
Beyond the CUDA fixes, this release expands platform availability with new build targets: Ubuntu s390x (IBM Z architecture), Windows arm64 CPU, and openEuler aarch64 with ACL Graph support for Ascend processors. Existing builds for macOS (Apple Silicon and Intel), Linux (x64/Vulkan/ROCm/OpenVINO/SYCL), Android arm64, Windows (x64 CUDA/Vulkan/HIP), and iOS XCFramework are also updated. This release is primarily a stability and compatibility update rather than introducing new features, but it ensures more reliable local LLM inference across diverse hardware configurations.
- Fixed CUDA context reset after reading memory size to prevent GPU memory errors
- Added new platform builds including Ubuntu s390x, Windows arm64 CPU, and openEuler aarch64 with ACL Graph
- Improved memory function with device mutex and exclusion of HIP/MUSA from buffer counting
Why It Matters
Improves stability for running LLMs locally on diverse hardware, especially CUDA systems.