DialogueSidon: Recovering Full-Duplex Dialogue Tracks from In-the-Wild Dialogue Audio
New AI model separates overlapping dialogue into clean, individual speaker tracks with faster inference.
A research team from the University of Tokyo and NTT has introduced DialogueSidon, a novel AI model designed to solve a critical bottleneck in speech processing. Most real-world two-speaker dialogue exists only as degraded, single-channel (monaural) audio where voices overlap, making it unusable for systems that require clean, isolated speaker tracks. DialogueSidon tackles this by performing joint restoration and separation, transforming messy, in-the-wild recordings into high-quality, full-duplex audio where each participant has their own clean track.
The model's architecture is a key innovation, combining a variational autoencoder (VAE) that compresses features from a self-supervised learning (SSL) speech model into a compact latent space, with a diffusion-based predictor that reconstructs the individual speaker representations from the mixed input. This hybrid approach allows DialogueSidon to not only separate the voices but also actively restore audio quality. In tests on English, multilingual, and real-world dialogue datasets, it substantially outperformed baseline methods in both intelligibility and separation metrics, while also delivering significantly faster inference times—a practical advantage for deployment.
- Jointly restores and separates degraded monaural dialogue into clean, speaker-specific tracks.
- Uses a VAE + diffusion model architecture on SSL speech features for efficient, high-quality output.
- Demonstrated substantial improvements in intelligibility and separation speed on real-world datasets.
Why It Matters
Unlocks vast libraries of noisy real-world conversations for training advanced speech AI, translation, and assistive tech.