Gemma 4 12B tool calling fix: custom chat template unlocks coding eval

Google’s Gemma 4 12B, a popular open-weight model aimed at developers, has faced widespread criticism for broken tool calling and coding support. Users reported that harnesses like OpenCode failed outright because the model’s default chat template didn’t format tool-call requests correctly. Now, a community fix has emerged: a custom jinja template file (custom-pub-chat-template-gemma4.jinja) that rewrites the conversation schema to match what the model actually expects. The solution, shared by u/HVACcontrolsGuru and amplified by u/boutell, requires compiling llama.cpp from source to enable the --jinja flag, then serving the GGUF quant (e.g., unsloth/gemma-4-12b-it-GGUF:UD-Q8_K_XL) with the template attached. Initial tests confirm that tool call bugs disappear, allowing developers to finally evaluate the model’s coding ability in real workloads.

This fix is critical because it separates genuine model capability from prompt engineering failures. Before the fix, many dismissed Gemma 4 12B as unusable for coding agents. Now, with proper tool calling, it can be compared fairly against Qwen 3 9B and other small models. The community cautions that results may still not be stellar, but at least the evaluation is valid. For professionals building AI-assisted coding tools or local LLM workflows, this workaround is essential for leveraging Gemma 4’s 12B parameter efficiency without integration headaches.