b8634

The open-source llama.cpp project, maintained by ggml-org, has released a significant update (commit b8634) that adds official support for IBM's Granite 4.0 model series. This update specifically addresses a critical issue where the previous implementation would break tool calling functionality—the ability for AI models to execute functions and interact with external systems. The fix introduces a new LLM_CHAT_TEMPLATE_GRANITE_4_0 template that correctly maps the assistant_tool_call role to the proper XML tag structure (<|start_of_role|>assistant<|end_of_role|><|tool_call|>) that Granite 4.0 models expect.

Without this fix, llama.cpp would emit the literal role name 'assistant_tool_call' which the model doesn't recognize, completely breaking tool calling functionality when not using Jinja templates. The update maintains backward compatibility by renaming the existing template to LLM_CHAT_TEMPLATE_GRANITE_3_X for older Granite 3.x models. The implementation includes automatic detection—if a template contains <|start_of_role|> followed by either <tool_call> or <tools>, it uses the new 4.0 template, otherwise it falls back to 3.x.

This release enables developers to run IBM's latest Granite 4.0 models locally across all major platforms including macOS (both Apple Silicon and Intel), Linux (with CPU, Vulkan, ROCm, and OpenVINO backends), Windows (with CPU, CUDA, Vulkan, SYCL, and HIP support), iOS, and openEuler systems. The commit was notably co-authored by Claude Opus 4.6, demonstrating AI-assisted development in action, and includes comprehensive tests for both 3.x and 4.0 template paths in both C++ and Jinja implementations.