Researchers crack neural codec training with new surrogate gradient method
SCALED method now interpretable, yielding up to 23.6% bitrate savings on x264.
Training neural pre- and post-processing networks to boost conventional video codecs has long been hampered by non-differentiable encoding decisions. A new paper from COMPACT (CNRS) at arXiv:2606.20671 provides a clean interpretation of the SCALED surrogate gradient method, showing it acts as a first-order local approximation of the codec rather than a reparameterization trick. This insight not only explains its effectiveness but also extends its applicability beyond simple downscalers to full neural wrapping pipelines.
The authors demonstrate that SCALED generalizes across diverse video codecs (x264 and VVenC), quality factors, and multiple downscaling ratios. Their approach yields BD-Rate (PSNR) reductions of up to -23.59% on x264 and -20.07% on VVenC relative to standard resampling baselines, without needing an additional network to mimic the codec. This makes it a practical drop-in solution for improving compression efficiency in real-world video delivery systems.
- SCALED gradient is reinterpreted as a first-order local approximation of video codecs, not a reparameterization trick
- Achieves up to -23.59% BD-Rate reduction on x264 and -20.07% on VVenC over standard resampling baselines
- Method works for full neural wrappers (pre/post processing) across multiple downscaling ratios and quality factors
Why It Matters
Clearer training method for neural codec wrappers could reduce streaming bandwidth costs by over 20% without hardware changes.