Developer Tools

llama.cpp fixes muse-glimmer bug that swallowed tool calls

A bug silently dropped tool calls across 43 turns in tau2-bench—now patched.

Deep Dive

llama.cpp contributor ruanslv committed a fix for a subtle bug in the Muse Glimmer chat template that caused tool calls to be silently swallowed into content. Muse Glimmer often answers a user and invokes a tool in a single generation. The template uses <|eom|> (end-of-message) when more messages follow in the same turn, and <|eot|> (end-of-turn) only at the very end. The previous parsing logic assumed the user-facing answer was always last, reading until <|eot|>. But when a tool call follows the answer, there's no <|eot|> before the call—so the parser consumed the tool markup as plain content, and no tool_calls were emitted. The tool never executed.

On a tau2-bench telecom run, this bug affected 43 turns across 19 of 114 tasks—roughly 17% of tasks, showing how common the pattern is. The fix changes the final-message rule to stop at <|eom|> and parse everything after as tool calls. The commit also adds models/templates/muse-glimmer.jinja and four parser tests covering: a plain answer, the <|eom|> junction, markup quoted inside an answer remaining content, and tool markup inside a to=self channel that should stay reasoning. Developers using llama.cpp with Muse Glimmer models should update to prevent silent tool failures.

Key Points
  • Bug in llama.cpp's Muse Glimmer template dropped tool calls after <|eom|> tokens, causing tools to never run
  • Affected 43 turns across 19 of 114 tasks in a tau2-bench telecom run (~17% of tasks)
  • Fix adds muse-glimmer.jinja template and 4 parser tests; changes parsing to stop at <|eom|> and treat trailing markup as tool calls

Why It Matters

This fix prevents silent tool failures in llama.cpp apps using Muse Glimmer, improving reliability for AI agents relying on tool calls.

📬 Get the top 10 AI stories daily