New TF-Restormer model handles mismatched audio input-output rates for speech restoration

A team of researchers has introduced TF-Restormer, a novel speech restoration model that tackles the challenge of mismatched input-output sampling rates—a common issue in real-world audio processing where recorded speech may have a different sample rate than the target clean output. Existing methods typically assume equal rates and rely on redundant resampling, which limits efficiency and performance. TF-Restormer formalizes this as the extended sampling-frequency-independent (xSFI) setting and uses a query-based asymmetric encoder-decoder architecture. The encoder processes only the observed input frequency band, while the decoder synthesizes the unobserved high-frequency band via extension queries with band-partitioned cross-attention. This asymmetric design allocates more capacity to analysis while keeping synthesis lightweight.

Training employs a scaled log-spectral loss, perceptual loss, and adversarial supervision via an SFI-STFT discriminator, enabling TF-Restormer as a single unified model to handle denoising, dereverberation, bandwidth extension, and combined distortions—all across multiple sampling rates. The model achieves state-of-the-art balance between fidelity and perceptual quality without any task-specific resampling. Accepted to ICML 2026, this work promises to simplify speech restoration pipelines for applications like hearing aids, voice assistants, and audio forensics, where input and output rates often vary.