New audio AI framework outperforms models in speech enhancement benchmarks
Researchers propose unified framework comparing 6 LM-based speech enhancement models, with continuous non-autoregressive approach leading performance gains
Researchers from TU Braunschweig and collaborators have developed a comprehensive framework to evaluate language model-based speech enhancement (SE) systems by rethinking their approach to neural audio codec latent spaces. The team, led by Yihui Fu and Zhengyang Li, analyzed six distinct paradigms including discrete/continuous autoregressive (D/CAR), discrete/continuous non-autoregressive (D/CNAR), discrete diffusion (DDiff), and continuous flow matching (CFM) models.
In benchmark tests using the URGENT 2025 Speech Enhancement Challenge dataset, all continuous-domain paradigms significantly outperformed their discrete counterparts. The continuous non-autoregressive approach (CNAR) emerged as the top performer across multiple evaluation metrics. The researchers further introduced an auxiliary loss fine-tuning strategy that delivered consistent improvements—ranging from 15-20%—in intrusive and non-intrusive quality metrics including DNSMOS, NISQA, PESQ, and POLQA.
- Six language model-based speech enhancement paradigms compared in unified framework using URGENT 2025 dataset
- Continuous non-autoregressive models (CNAR) outperformed all discrete approaches by significant margins
- New auxiliary loss fine-tuning improved quality metrics like PESQ and DNSMOS by 15-20%
Why It Matters
This research sets new benchmarks for AI-powered audio enhancement, enabling clearer voice communications for applications from teleconferencing to hearing aids.