KV cache grafting lifts Gemma 4 12B accuracy from 76.7% to 90%

A new research paper from Google presents byte-exact KV cache grafting, a method that allows frozen large language models to incorporate verified knowledge without retraining. By storing knowledge as key-value state and restoring it byte-identically to fresh computation, the technique effectively augments the model’s existing capabilities. Applied to Google’s Gemma 4 12B model, the approach targets the AIME 2025 routing benchmark, where accuracy jumped from 76.7% to 90.0%. This demonstrates that frozen models can be significantly improved by caching verified reasoning paths, preserving exactness while avoiding the cost and risks of fine-tuning.

The work has broad implications for deploying LLMs in production, as it allows practitioners to inject domain-specific or verified knowledge into a frozen model without modifying weights or requiring additional compute for retraining. The byte-identical restoration ensures no degradation of the original model’s performance. The authors will present their findings at the AGI Summit on July 19, and the full paper is available on arXiv (2607.14431). This approach could be particularly valuable for applications requiring high reliability, such as legal, medical, or scientific reasoning, where preserving exact knowledge is critical.