Elastic Time gives neural audio codecs dynamic frame rates for better efficiency

Neural audio autoencoders are central to compression, feature extraction, and generation, but most models operate at a fixed latent frame-rate, allocating the same temporal budget to both information-dense and sparse regions. This leads to unnecessarily long sequences and wasted compute. Elastic Time, developed by researchers Dimitrios Bralios, Paris Smaragdis, and Minje Kim, tackles this by introducing a dynamic frame-rate bottleneck that converts fixed-frame-rate autoencoders into adaptive ones. The method learns a lightweight latent predictor that decides which frames can be safely skipped at inference time, with a greedy boundary selection process that reconstructs skipped frames efficiently. This allows the model to dynamically adjust temporal resolution based on information density, achieving better efficiency-quality tradeoffs than static baselines.

Beyond compression, Elastic Time's flexible rate control at deployment time opens up new possibilities for downstream modeling. By reducing sequence length for low-information segments, it enables more efficient training and inference for audio generation tasks and long-context models that must process extended audio segments. The technique is demonstrated to work with existing autoencoder architectures, making it a practical plug-in upgrade. Presented at Interspeech 2026, this work addresses a fundamental inefficiency in neural audio coding—potentially reducing latency and memory usage for real-time applications and large-scale speech or music processing systems.