New Residual Coders Boost Learned Compression by 30-60% for Scientific Data

Lossy compression is critical for handling the enormous spatiotemporal datasets generated by scientific simulations. Existing Guaranteed Autoencoder (GAE) methods preserve accuracy by adding per-block residual corrections, but in the high-fidelity regime (NRMSE 10^-6 to 10^-4) the correction stream dominates the bitrate, negating the advantages of learned compression. In a new arXiv preprint, researchers Liangji Zhu, Sanjay Ranka, and Anand Rangarajan propose a residual-centric approach: they argue that the learned residual has a different structure than the original field and should be coded with a dedicated representation.

To that end, they introduce two residual coders. LBRC is a deterministic, training-free pipeline that adaptively quantizes the residual to the target NRMSE, then losslessly encodes the integer result using 3D Lorenzo differencing, zigzag mapping, bit-plane coding, and entropy coding. NGLR adds a causal neural predictor that outputs a normalized bias for an integer-rounded Lorenzo prediction, reducing entropy while preserving deterministic decoding. On E3SM, JHTDB, and ERA5 datasets, LBRC improves compression ratios by 30-60% over GAE and is competitive with SZ, while NGLR adds a further 10-40% gain, outperforming SZ across the high-fidelity range.