Gemma 4 QAT vs PTQ: Missing benchmarks spark community debate

A Reddit thread from user Character_Split4906 has ignited discussion in the AI community over the lack of direct benchmarks comparing Google's Gemma 4 models quantized to 4-bit using Quantization-Aware Training (QAT) via Unsloth against standard 8-bit quants using Post-Training Quantization (PTQ). The user notes that QAT is known to retain high accuracy relative to the original BF16 precision, but wonders how a 4-bit QAT model actually performs in practice versus a more traditional 8-bit PTQ—a comparison crucial for those balancing model size, speed, and quality.

The thread reveals that while some users have reported anecdotal results—suggesting that 4-bit QAT models can match or even exceed 8-bit PTQ in certain tasks—no comprehensive, standardized evaluation has been published. This gap is notable because QAT requires additional training time (the model is fine-tuned with quantization awareness) but can yield significantly smaller models with less degradation, potentially enabling deployment on consumer hardware. The lack of hard numbers leaves practitioners guessing: is the extra effort of QAT worth it compared to simply using 8-bit PTQ? As local LLM deployment grows, this benchmark is urgently needed.