Spatial conditioning outperforms beamforming in speaker diarization

A new study from researchers at Paderborn University investigates how to best integrate spatial information from multi-channel audio into foundation-model-based speaker diarization. Diarization – identifying who spoke when – is critical for meeting transcription and conversation analysis. Large pretrained single-channel models like WavLM already achieve state-of-the-art performance, but real-world recordings often come from microphone arrays. The team compared three integration strategies: (1) a cascade of a beamformer followed by a single-channel WavLM, (2) a multi-channel version of WavLM, and (3) a single-channel WavLM whose downstream network is conditioned on explicit spatial features (e.g., inter-channel phase and level differences).

Results show that conditioning is the clear winner. Surprisingly, the beamformer front-end actually degraded diarization accuracy during overlapped speech, likely because it mixes voices together. The multi-channel foundation model performed well but required heavy retraining. The conditioning approach, on the other hand, achieved the lowest error rates and was more robust. A detailed error analysis revealed that combining spectral and spatial features removed many errors that occurred when using either modality alone. The work was accepted at IWAENC 2026 and provides practical guidance for building better meeting diarization systems that leverage spatial audio without sacrificing the power of pretrained models.