AMD-Led Research Fixes 4-Bit Quantization Flaw for Faster LLM Inference

Standard symmetric integer quantization for neural networks suffers from an asymmetry in the signed integer alphabet: it has one more negative representable value than positive. By convention, the scale is strictly positive, so this extra value sits on the negative tail, forcing clipping of positive outliers. At low bit widths (e.g., 4-bit), this clipping becomes a significant source of quantization error. Asymmetric quantization fixes this by adding a zero-point shift, but that flexibility carries a runtime penalty — on an AMD EPYC 'Turin' CPU, a 4-bit symmetric format uses up to 9% less memory with up to 2.45x higher throughput than its asymmetric counterpart.

The researchers propose an alternative: signed symmetric quantization with a 'signed absmax grid.' This method retains the runtime profile of symmetric quantization (no zero-point) but places the extra representable value on the dominant outlier tail via a lightweight sign selection rule. They prove that this grid is conditionally bound-optimal on ℓ₂ quantization error, and show the condition holds for 88–99% of weight groups across pre-trained LLMs at low bit widths. Empirically, models from the Qwen3, Qwen3.5, and Llama3 families show improved perplexity and downstream few-shot accuracy over standard unsigned symmetric quantization — all with zero additional inference cost.