Moving Speaker Separation via Parallel Spectral-Spatial Processing

A research team from Tampere University and Aalto University has published a breakthrough paper on IEEE Transactions on Audio, Speech and Language Processing titled 'Moving Speaker Separation via Parallel Spectral-Spatial Processing.' The work introduces a novel dual-branch neural network architecture called PS2 that fundamentally rethinks how to handle the challenging problem of separating overlapping speech from moving sources. Traditional methods force a single network stream to model both the spectral (sound frequency) and spatial (source location) features of audio, creating a modeling conflict as these features evolve at different temporal scales. The PS2 architecture solves this by processing these features in parallel, dedicated streams before intelligently fusing them.

The technical innovation lies in the parallel design: a spectral branch uses a BLSTM-based frequency module, a Mamba-based temporal module, and self-attention, while a spatial branch uses bi-directional GRUs to track the geometric relationships between speakers and microphones. A cross-attention fusion mechanism dynamically weights the contributions from each branch. This approach yielded a significant 1.6-2.2 dB improvement in the Scale-Invariant Signal-to-Distortion Ratio (SI-SDR) over previous state-of-the-art methods on datasets like WHAMR! and a new WSJ0-Demand-6ch-Move dataset. Crucially, the model maintains robust performance (over 13 dB SI-SDR improvement) under varying reverberation, noise, and—most importantly—different source movement speeds, a key weakness of prior models. This research paves the way for vastly more effective speech separation in real-world dynamic settings, from video conferencing and smart speakers to hearing aids and autonomous vehicles.