Llama.cpp b9656 hardens tool call parsing with OpenAI-style support
Open-source LLM runtime now gracefully handles OpenAI tool calls with cleaner errors.
The open-source llama.cpp project, with over 117k stars and 19.6k forks on GitHub, has released version b9656 with significant improvements to tool call parsing. The update hardens the peg-native parser by accepting an optional leading 'type: function' field, enabling OpenAI-style tool calls to parse correctly on templates that serialize with the name field first. On final parse failures, the parser now returns a clean error and logs the unparsed fragment instead of throwing raw positions and input. Additionally, the raw arguments string is preserved in func_args_not_string when it isn't valid JSON, preventing the prompt render from aborting. These changes are minimal and do not affect behavior on successful parses, while greatly improving debugging and reliability for developers using tool-calling LLMs locally.
Beyond parsing improvements, the release also gates the OpenAI wrapper grammar trigger via an autoparser flag, ensuring that the leading 'type: function' leniency is only applied when the openai_wrapper_trigger is set via the analysis flag from the generator to build_json_tools_flat_keys. This release is built for a wide range of platforms, including macOS (Apple Silicon with and without KleidiAI, Intel x64), iOS XCFramework, Linux (x64/arm64/s390x CPU, Vulkan, ROCm 7.2, OpenVINO, SYCL), Android (arm64 CPU), and Windows (x64/arm64 CPU, CUDA 12/13, Vulkan, SYCL, HIP). This extensive platform support makes the improved tool call handling accessible to nearly any local LLM deployment scenario.
- Peg-native tool call parser now accepts optional 'type: function' field for OpenAI-style compatibility.
- On parse failures, returns clean error messages and logs unparsed fragments instead of raw positions.
- Preserves invalid JSON strings in func_args_not_string to avoid aborting prompt render.
Why It Matters
Enables reliable local LLM tool usage with OpenAI-compatible function calling, improving developer experience and debugging.