Developer Tools

llama.cpp b9590 fixes json_schema bug in LFM2/LFM2.5 templates

LFM2/LFM2.5 now respects json_schema for structured output – a critical fix for developers.

Deep Dive

ggml-org has released llama.cpp b9590, a significant bugfix update for the popular local LLM inference library. The key fix addresses an issue in the chat template handlers for LFM2 and LFM2.5 (likely referring to Llama 2 and Llama 2.5 variants, or similar model families like LlaMA 2). Previously, these specialized template handlers only built a grammar for tool-calling when processing the response_format parameter, but they silently ignored the json_schema field. This meant that any structured output requirements specified via json_schema were not enforced during generation, potentially leading to invalid or malformed responses for applications relying on structured data.

This release comes with a comprehensive set of pre-built binaries covering multiple platforms and hardware backends. For macOS users, options include Apple Silicon (arm64), Apple Silicon with KleidiAI acceleration, Intel (x64), and iOS XCFramework. Linux builds cover Ubuntu x64 and arm64 (CPU), as well as accelerated backends like Vulkan, ROCm 7.2, OpenVINO, and SYCL FP32. Windows users get CPU, CUDA (versions 12 and 13), Vulkan, SYCL, and HIP builds. Android arm64 is also supported. Additionally, openEuler builds (with Ascend ACL Graph acceleration) are included. The release also updates UI assets. This diverse build matrix ensures that developers and users on various configurations can immediately benefit from the fix.

Key Points
  • Fixes LFM2/LFM2.5 template handlers ignoring json_schema from response_format during tool-call grammar construction.
  • Available for macOS, Linux, Windows, Android, iOS with backends including CPU, Vulkan, CUDA 12/13, ROCm, OpenVINO, SYCL, and HIP.
  • Includes UI assets update alongside platform-specific build fixes.

Why It Matters

Ensures reliable structured output generation for apps using LFM2/LFM2.5 templates in llama.cpp.

📬 Get the top 10 AI stories daily