llama.cpp Just Made a Sneaky Upgrade to Its Grammar Engine — Here's Why That's a Big Deal
New release speeds up structured output with AC automaton-based grammar parsing...
The popular local LLM runtime llama.cpp has shipped release b9744, which introduces a major refactor of its grammar generation pipeline. The team rewrote the common/peg module to first parse grammars into GBNF format, then compile them into an AC (Aho-Corasick) automaton. This approach allows for efficient multi-string matching during inference, significantly speeding up grammar-constrained generation (e.g., JSON output, code syntax).
The release includes comprehensive tests with multiple strings, padding optimizations, and a regression fix in server-tools.cpp. Builds are available for all major platforms: macOS (Apple Silicon, Intel, iOS), Linux (x64, arm64, Vulkan, ROCm, OpenVINO, SYCL), Windows (CPU, CUDA, Vulkan, OpenCL, HIP), and Android arm64. Users can now generate structured outputs faster and with fewer errors.
- Refactored grammar generation from PEG to GBNF then into an AC automaton for speed
- Includes performance optimizations like state padding and inline string handling
- Available across 20+ platforms including Windows, macOS, Linux, iOS, and Android
Why It Matters
Local LLM inference gets faster structured output, enabling reliable JSON/code generation without cloud dependency.