b9038

The open-source community behind llama.cpp has released version b9038, a minor but impactful update that refines how the tool estimates available GPU memory when using OpenCL. Previously, memory estimates for the --fit option could be inaccurate, leading to failed model loads or crashes. Now, the code explicitly queries CL_DEVICE_GLOBAL_MEM_SIZE to get a reliable real-time snapshot of available VRAM. This change was contributed and signed off by Florian Reinle, and the release includes 30 pre-built binary assets covering macOS (Apple Silicon and Intel), Linux (x64, arm64, Vulkan, ROCm, OpenVINO, SYCL), Windows (CPU, CUDA 12/13, Vulkan, SYCL, HIP), Android (arm64), and openEuler.

The --fit flag in llama.cpp is critical for users who want to automatically load the largest model that fits into their GPU memory. With this fix, users running on OpenCL-capable hardware—such as older AMD GPUs, Intel integrated graphics, or devices that lack native CUDA support—will see more predictable behavior and fewer load failures. The update doesn't introduce new features, but it improves stability for a large segment of the user base that relies on OpenCL for local inference. As the local AI movement grows, such incremental reliability improvements help make self-hosted models more accessible on diverse hardware.