Columbia researchers fix AI speech suppression with inference-time fix
New rethink-and-refine correction module cuts speech distortion by 30% without retraining...
A rethink-and-refine correction module tackles over-suppression in speech enhancement models, where speech gets removed alongside noise. It works entirely at inference time, needing no extra training, so it can plug into diverse enhancement systems. Using an ASR model to get word- or phoneme-level alignments, it spots unreliable intervals and selectively remixes them via convex interpolation, with per-segment weights tuned to balance perceptual quality and speech preservation. Tests on the URGENT 2024/2025, VCTK-DEMAND, and MSP-PODCAST datasets show consistent gains in perceptual quality, intelligibility, and downstream performance over standard enhancement alone, highlighting the framework's value for robust speech processing.
- Columbia University teamβs rethink-and-refine module fixes over-suppression in speech enhancement models without retraining
- Achieves 15β25% better perceptual quality and 30% less speech distortion on URGENT 2024/2025, VCTK-DEMAND, and MSP-PODCAST datasets
- Operates entirely at inference time via word/phoneme-level alignment and selective remixing
Why It Matters
Delivers clearer, more natural voice processing for call centers, hearing aids and voice assistants without costly model retraining.