UniPASE: A Generative Model for Universal Speech Enhancement with High Fidelity and Low Hallucinations

A research team led by Xiaobin Rong has introduced UniPASE, a generative model designed for universal speech enhancement (USE). The system tackles the complex task of restoring speech from a wide array of distortions—like background noise, reverb, or compression artifacts—across multiple audio sampling rates. At its core is a novel module called DeWavLM-Omni, which is fine-tuned from the powerful WavLM foundation model using knowledge distillation on a massive, supervised dataset of distorted audio. This module's key innovation is directly converting a noisy input waveform into a clean, linguistically accurate phonetic representation, a process that ensures the enhanced speech remains faithful to the original words and drastically reduces the risk of the AI "hallucinating" incorrect sounds or syllables.

Following this phonetic cleanup, UniPASE uses an Adapter to generate detailed acoustic features, which a neural vocoder then turns into a high-fidelity 16kHz waveform. A final PostNet module upscales this to 48kHz before resampling it back to the input's original rate, enabling seamless handling of any audio file. The model's performance is not just theoretical; it served as the backbone for the team's submission to the prestigious URGENT 2026 Challenge, where it secured first place in the objective evaluation, outperforming other state-of-the-art models. The availability of source code and audio demos allows developers and audio engineers to test its capabilities for applications from podcast cleanup to real-time communication.