Gemma 4 QAT models deliver 45% faster inference on AMD 7900 XTX

Google's Gemma 4 family now includes quantization-aware training (QAT) versions, which promise to retain the fidelity of BF16 weights while using Q4 precision. A developer benchmarked these models against standard quantizations (Q8_0, UD-Q4, Q4_K_M) on a single AMD 7900 XTX via ROCm and llama-swap at temperature 1.0, with no token cap. The standout was the 12B QAT model: total generation time dropped from 323 seconds to 176 seconds (45% faster), throughput improved 83%, and VRAM usage fell by 5.7GB. Quality remained identical across all prompts, and on a constraint-following task the QAT model finished in 24 seconds versus 124 seconds for the regular version.

The 26B QAT model over UD-Q4 delivered moderate 1.0-1.38x speedups and saved 2GB VRAM with no quality degradation. The 31B QAT over Q4_K_M was 1.3-1.5x faster and produced 8% more output — on creative continuation, the regular model stopped at 710 characters while QAT continued to 1256. The E4B model showed confounding results due to bit-width differences and needs same-precision testing. For professionals running large language models locally on consumer hardware, these QAT models offer a frictionless path to faster inference and larger context windows without sacrificing output quality.