New ECC neural codec cuts speech bitrate by 76.3% over baselines

Researchers from Shanghai Jiao Tong University and other institutions have introduced ECC (Entropy-Constrained Codec), a neural speech compression framework that tackles a key limitation in existing learned codecs: the decoupling of representation learning from probability modeling. While many neural codecs use preset-rate discrete symbols or apply entropy coding only after symbol generation, ECC integrates a learned entropy model directly into the training and inference pipeline. It combines scalar quantization with hyperprior-based side information, channel-wise context modeling, latent residual prediction, and lightweight temporal modeling to estimate likelihoods for rate estimation during training and arithmetic coding during inference.

ECC also introduces an “entropy skip” mechanism that omits highly predictable residual symbols using decoder-available scale estimates, eliminating the need to transmit additional skip masks. Extensive experiments show ECC achieves a favorable low-bitrate rate–distortion trade-off, reducing BD-rate by 39.9% on the ViSQOL metric and 76.3% on PESQ over conventional and neural codec baselines across two widely used test sets. Ablation studies confirm the effectiveness of each entropy modeling component. The project page and code are available online, marking a significant step toward more efficient speech compression for real-world applications like voice calls and streaming.