b8368
Latest commit patches critical 'disable reasoning' flaw while adding Vulkan, ROCm, and OpenVINO GPU backends.
The open-source project Llama.cpp, maintained by ggml-org, has pushed a significant update with commit b8368. This release primarily addresses a critical bug (#20606) in the command-line interface (CLI) where the `--disable-reasoning` flag was not functioning correctly. The fix ensures users can properly control the reasoning capabilities of running models, a crucial feature for debugging and optimizing inference workflows. The commit is verified with GitHub's GPG signature, confirming its authenticity from the core maintainers.
The update is not just a bug fix; it substantially broadens hardware compatibility. New pre-built binaries now support Windows x64 with Vulkan API acceleration and Linux with ROCm 7.2 for AMD GPUs and OpenVINO for Intel hardware. This expands the ecosystem beyond the established CUDA (NVIDIA) and Apple Metal backends. For developers, this means more flexible deployment options for running models like Llama 3 or Mistral directly on CPUs or a wider array of GPUs, enhancing performance and accessibility.
- Fixes CLI bug #20606 that broke the `--disable-reasoning` flag, restoring control over model behavior.
- Adds new GPU acceleration backends: Windows Vulkan, Linux ROCm 7.2 (AMD), and Linux OpenVINO (Intel).
- Provides verified, pre-built binaries for macOS (Apple Silicon/Intel), iOS, Linux, Windows, and openEuler.
Why It Matters
This patch stabilizes a core tool for millions running local LLMs and unlocks faster inference on non-NVIDIA hardware, democratizing access.