G-MaP-SE: New GMM-based prior matching boosts speech enhancement without clean audio

Traditional speech enhancement models often condition on speaker embeddings extracted from clean enrollment audio—but that clean audio isn't always available at inference. Some methods try using noisy embeddings directly, but those degrade under heavy noise or domain shift. G-MaP-SE solves this by first building a Gaussian Mixture Model (GMM) prior from clean speech embeddings during training. At inference, it takes a noisy conditioning embedding and matches it to the closest component of that GMM prior, effectively 'denoising' the conditioning signal without needing extra enrollment. This matched prior is then fused into a convolutional time-frequency enhancement backbone via a lightweight gated fusion module that adaptively weights features.

The results are striking. On VoiceBank+DEMAND, G-MaP-SE achieves PESQ scores that are 0.2 points higher than standard noisy-conditioned baselines, and on DNS Challenge 2020 it reduces the gap to the oracle clean-conditioned upper bound by over 50% in terms of STOI. The method requires no enrollment audio at inference, works with any pretrained speaker embedding extractor, and is computationally lightweight—the gated fusion adds only ~5% extra parameters. The team also released code, audio samples, and checkpoints. Accepted to Interspeech 2026, this work offers a practical path to robust speech enhancement in real-world call centers, hearing aids, and voice assistants where clean enrollment is unavailable.