The One Llama.cpp Update That Makes AI Outputs Suddenly Reliable
New release enforces JSON schema on LLM responses, improving reliability for developers.
The open-source llama.cpp project (ggml-org) has released version b9707, introducing a major new feature for its inference server: schema validation support. This allows developers to attach a JSON schema to the server's completion or chat endpoints, forcing the LLM to generate outputs that conform to a predefined structure. The feature is implemented via a new "schema" field in the server API, alongside error messages that now include the exact field name that failed validation. This is a significant quality-of-life improvement for anyone building structured pipelines, such as function calling, RAG output formatting, or automated data extraction.
The release b9707 is already available across all major platforms: macOS (Apple Silicon and Intel, including KleidiAI-optimized for arm64), Linux (CPU, Vulkan, ROCm 7.2, OpenVINO, SYCL), Windows (CPU, CUDA 12/13, Vulkan, HIP), Android (arm64), and iOS via XCFramework. The build matrix reflects the project's broad device support. This update does not change the underlying model inference speed but adds critical robustness for production deployments. Developers can now ensure that every response from the LLM adheres to a contract, reducing parsing errors and downstream failures.
- Server now accepts a 'schema' parameter to enforce JSON output structure on model responses.
- Validation error messages include the failing field name, improving debugging in production.
- Supports all major platforms: macOS (Apple Silicon/Intel), Linux (CPU/Vulkan/ROCm/OpenVINO/SYCL), Windows (CPU/CUDA/Vulkan/HIP), Android, and iOS.
Why It Matters
Developers can now enforce structured output from any local LLM, reducing errors in automated pipelines.