New AI method separates moving sounds using mutual localization and separation
Joint tracking and separation boosts accuracy for moving sound sources in reverberant rooms.
A new paper by Donghang Wu, Xihong Wu, and Tianshu Qu introduces a technique that links sound source localization and separation for moving sources. Traditional methods either need prior direction-of-arrival (DOA) knowledge or use noisy localization estimates, hurting performance when sources move. The proposed approach starts with an initial tracking stage that estimates source trajectories from the mixture using envelope cues. Then a mutual facilitation loop iteratively performs separation based on current trajectories and re-tracks the separated signals to refine trajectories, improving both tasks. A final neural beamformer uses the refined trajectories and multi-channel outputs to generate precise single-channel separation.
Simulations under reverberant conditions with moving speakers showed that this iterative refinement significantly boosts separation accuracy compared to baselines. The work, published in IEEE TASLP, demonstrates that treating localization and separation as jointly optimized problems yields better results for dynamic auditory scenes. This could enhance applications like hearing aids, autonomous drones, and voice-controlled systems where sources move unpredictably. The code and data are not yet publicly available, but the paper provides a strong theoretical and empirical foundation for future research in moving sound source separation.
- Proposes mutual facilitation: iterative cycles of separation and tracking refine both tasks for moving sources.
- Three-stage pipeline: initial envelope-based tracking, iterative refinement loop, then neural beamformer for final separation.
- Outperforms existing methods in reverberant, moving-source simulations; published in IEEE TASLP.
Why It Matters
Enables robust sound separation for drones, hearing aids, and autonomous systems in dynamic environments.