llama.cpp b9567 fixes HTTP header flushing in server mode
This patch prevents parsing errors during HTTP response flushing...
The latest release of llama.cpp, version b9567, addresses a critical bug in the server component. Previously, the server would attempt to parse HTTP headers while they were being flushed to the client, leading to potential parsing errors, crashes, or malformed responses. The fix, referenced as issue #24281, ensures that the server no longer parses headers during the flush operation, improving reliability for all users running the HTTP inference server.
This maintenance release continues llama.cpp's tradition of broad platform support. Builds are available for macOS (Apple Silicon with optional KleidiAI, Intel, and iOS XCFramework), Linux (x64/arm64 with CPU, Vulkan, ROCm 7.2, OpenVINO, SYCL FP32), Android (arm64 CPU), and Windows (x64/arm64 CPU, CUDA 12/13, Vulkan, SYCL, HIP). Notably, some builds like macOS KleidiAI, openEuler, and Windows SYCL are disabled in this release. The UI assets have also been refreshed, likely fixing minor visual glitches for users of the built-in web interface.
- Fixes HTTP header parsing during flushing in the server component (#24281), preventing crashes
- Available across 20+ platform builds including macOS, Linux, Windows, Android, and iOS
- Includes UI assets update; some experimental builds (KleidiAI, openEuler, SYCL) are disabled
Why It Matters
For developers self-hosting LLMs, this fix stabilizes the inference server, reducing errors during high-throughput requests.