Quantization cuts data center energy costs by 34% without reducing LLM output

A new paper from Bojun Du, Xiaoyi Fan, Ershun Du, and colleagues introduces a quantization-enabled energy management framework for data centers running large language model (LLM) inference workloads. The core insight: model quantization—typically used for latency and memory improvements—can serve as a flexibility lever for grid demand response (DR). By adjusting the precision of model weights during inference, data centers can dynamically modulate power consumption without dropping requests. The authors build a quantization-to-power model that maps each model and quantization configuration to a set of dispatchable parameters, enabling precise energy adjustments.

The framework operates in two stages: first, it selects optimal model instances and precision levels; second, it routes requests and co-optimizes across multiple campuses using grid-side electricity and carbon signals. In case studies, the approach cut total data-center operating costs by 34.3% while maintaining the same volume of served tokens. This demonstrates that quantization offers untapped flexibility for energy management, addressing tightening grid constraints without sacrificing LLM throughput. The work is particularly relevant as LLM inference loads grow rapidly and data centers face increasing demand response requirements.