DiariZen Explained: A Tutorial for the Open Source State-of-the-Art Speaker Diarization Pipeline

Nikhil Raghav published DiariZen, an open-source speaker diarization pipeline that sets a new state-of-the-art across multiple benchmarks. The system uses a structurally pruned WavLM-Large encoder, a Conformer backend with powerset classification, and VBx clustering with PLDA scoring to accurately determine 'who spoke when' in multi-speaker audio streams. This tutorial paper breaks down the entire pipeline into seven distinct stages: (1) audio loading and sliding window segmentation, (2) WavLM feature extraction with learned layer weighting, (3) Conformer backend and powerset classification, (4) segmentation aggregation via overlap-add, (5) speaker embedding extraction with overlap exclusion, (6) VBx clustering with PLDA scoring, and (7) reconstruction and RTTM output. Each stage includes conceptual motivation, source code references, intermediate tensor shapes, and annotated visualizations from a 30-second AMI Meeting Corpus excerpt.

The implementation is fully open-source, with standalone executable scripts for each block and a Jupyter notebook for end-to-end execution. This addresses a key pain point: the DiariZen architecture previously spanned multiple repositories and frameworks, making it difficult for researchers to understand, reproduce, or extend. By providing a self-contained tutorial with 13 pages, 7 figures, and 2 tables, Raghav enables practitioners to adopt and build upon this state-of-the-art system. The pipeline's hybrid approach combines the best of end-to-end neural diarization (EEND) with classical clustering, achieving superior performance while remaining accessible for real-world applications like meeting transcription, podcast analysis, and voice-based analytics.