SEMamba++: A General Speech Restoration Framework Leveraging Global, Local, and Periodic Spectral Patterns
New architecture beats multiple baselines by tailoring State-Space Models to speech's unique spectral characteristics.
Researchers Yongjoon Lee and Jung-Woo Choi have proposed SEMamba++, a new AI framework designed to tackle the complex challenge of general speech restoration. The work, submitted to Interspeech 2026, addresses a key limitation in existing State-Space Models (SSMs) like SEMamba: while powerful for denoising, they aren't inherently optimized for the critical, structured patterns of human speech, such as spectral periodicity and multi-resolution frequency content. SEMamba++ introduces architectural innovations that bake these speech-specific features directly into the model as inductive biases.
The core of the framework is a novel component called Frequency GLP (Global, Local, Periodic), a feature extraction block designed to efficiently leverage the properties of frequency bins in audio spectrograms. Alongside this, the team designed a multi-resolution parallel time-frequency dual-processing block to capture diverse spectral patterns across different scales. A final learnable mapping module further refines the output. The authors report that the combined system, SEMamba++, achieves the best performance among multiple baseline models for speech restoration tasks, all while maintaining computational efficiency—a crucial factor for real-world deployment.
- Introduces Frequency GLP block for efficient, speech-optimized frequency feature extraction from spectrograms.
- Uses a multi-resolution parallel architecture to capture global, local, and periodic spectral patterns critical for speech.
- Achieves state-of-the-art performance on speech restoration benchmarks while remaining computationally efficient.
Why It Matters
Enables clearer voice calls, audio archives, and assistive tech by removing noise and distortion more effectively than current AI models.