New bias correction cuts KV-cache memory 50% in video diffusion models

Chunk-wise autoregressive video diffusion models store a KV cache of previous chunks to avoid redundant computation, but as videos grow longer, this cache rapidly becomes a memory bottleneck. Quantizing the cache to low bitwidths reduces memory pressure but typically harms video quality. The authors show the key driver is a systematic bias in attention weights: due to the convexity of the exponential in softmax attention, quantization noise inflates the contribution of cached keys—a phenomenon they term the Jensen bias. This causes quantized keys to steal attention mass from the unquantized current chunk, degrading output fidelity.

The team proposes a per-attention-score correction that removes this bias in expectation, computed on the fly from the quantization step sizes of the cached keys and the query norm. The method uses a second-order Taylor approximation, making the computational overhead negligible and requiring no additional memory beyond the cache. Tested on MAGI-1, SkyReels-V2, and HY-WorldPlay at INT2 quantization, the correction recovers most of the quality lost to aggressive quantization, achieving near-BF16 video quality. It even outperforms INT4 quantization while using 50% less memory, making it a practical solution for long-video generation.