b8719
A critical fix in the popular llama.cpp library plugs a persistent memory leak discovered during AI training.
The maintainers of llama.cpp, the widely-used C++ inference engine for models like Llama 3, have patched a significant memory leak in their latest commit (b8719). The bug was located in the `ggml_opt_free` function, which is responsible for cleaning up memory after optimization steps during AI model training. Specifically, the function was failing to free the `ctx_copy` context object, a data structure created during the graph allocation process for training. This oversight meant that every time a training session concluded, approximately 900 KB of memory—the size of a typical graph context in projects like sindarin-pkg-tensor—remained allocated and unreachable, creating a persistent leak.
The issue was identified using AddressSanitizer, a tool for detecting memory errors. The fix is simple but essential: the code now properly calls `ggml_free()` on the `ctx_copy` pointer. This change is guard-free, meaning it safely handles cases where the pointer is null. For developers and researchers using llama.cpp for on-device training or fine-tuning of models, this patch directly translates to improved system stability, especially during extended or batch training jobs where memory leaks can compound and lead to crashes or performance degradation. The update has been rolled out across all supported platforms, including macOS, Linux, Windows, and iOS, ensuring broad compatibility for the library's significant user base.
- Fixes a memory leak in `ggml_opt_free` that failed to free the `ctx_copy` context.
- Prevents a leak of ~900 KB of memory per AI model training session, as identified in the sindarin-pkg-tensor project.
- Patch (commit b8719) is now live across all supported platforms including CPU, CUDA, Vulkan, and ROCm backends.
Why It Matters
This fix is critical for developers running stable, long-term AI training sessions locally, preventing memory exhaustion and crashes.