Yang & Bajić's bit allocation transfer cuts video bitrate 20% with perceptual quality

In a new arXiv paper (2608.07799), Runyu Yang and Ivan V. Bajić tackle a long-standing problem: traditional block-based video codecs like H.264/AVC, H.265/HEVC, and H.266/VVC rely on hand-crafted Rate-Distortion Optimization (RDO) that minimizes Mean Squared Error (MSE), which often mismatches human perception. While neural video compression can optimize perceptual metrics like MS-SSIM directly, its heavy computational cost limits real-world use. The authors bridge this gap with a bit allocation transfer framework.

The approach trains a quantization step generation model using perceptual loss inside the neural codec DCVC-FM. This model takes an original frame and a motion-compensated prediction, and outputs a quantization step map. The map is then converted into block-wise bit ratios and ultimately a Quantization Parameter (QP) map for a standard video encoder. Because the QP map feeds into existing encoders like HM-16.20 (HEVC) or VTM-23.0 (VVC), the core decoding syntax remains untouched—meaning any standard decoder can play the resulting bitstream without modification. On the HEVC B~D dataset, the method achieves 20.20% bitrate savings over JM-19.0 (H.264), 8.25% over HM-16.20 (HEVC), and 8.37% over VTM-23.0 (VVC), with additional gains when predicted frames are used. This effectively lets neural networks teach traditional codecs where to spend bits for better perceptual outcomes, without the latency or complexity of full neural decoding.