Developer Tools

llama.cpp b9567 fixes HTTP header flushing in server mode

This patch prevents parsing errors during HTTP response flushing...

Deep Dive

The latest release of llama.cpp, version b9567, addresses a critical bug in the server component. Previously, the server would attempt to parse HTTP headers while they were being flushed to the client, leading to potential parsing errors, crashes, or malformed responses. The fix, referenced as issue #24281, ensures that the server no longer parses headers during the flush operation, improving reliability for all users running the HTTP inference server.

This maintenance release continues llama.cpp's tradition of broad platform support. Builds are available for macOS (Apple Silicon with optional KleidiAI, Intel, and iOS XCFramework), Linux (x64/arm64 with CPU, Vulkan, ROCm 7.2, OpenVINO, SYCL FP32), Android (arm64 CPU), and Windows (x64/arm64 CPU, CUDA 12/13, Vulkan, SYCL, HIP). Notably, some builds like macOS KleidiAI, openEuler, and Windows SYCL are disabled in this release. The UI assets have also been refreshed, likely fixing minor visual glitches for users of the built-in web interface.

Key Points
  • Fixes HTTP header parsing during flushing in the server component (#24281), preventing crashes
  • Available across 20+ platform builds including macOS, Linux, Windows, Android, and iOS
  • Includes UI assets update; some experimental builds (KleidiAI, openEuler, SYCL) are disabled

Why It Matters

For developers self-hosting LLMs, this fix stabilizes the inference server, reducing errors during high-throughput requests.

📬 Get the top 10 AI stories daily