Researchers Fix a Big Flaw in AI Meeting Notes: Missing Words

Speaker diarization models — the systems that figure out who spoke when — are often trained on loose labels like speech segments with padded boundaries or filled pauses, and they tend to produce similarly loose outputs. A proposed fix: pseudo-labeling based on the averaged outputs of causal and anticausal models, which yields tighter boundaries. But because those pseudo-labels are estimation-based, they can suffer from over-tightening, increasing missed detections that can propagate as unrecoverable errors to downstream tasks. This paper analyzes the causes of over-tightening and proposes three approaches: removing pause filling rather than padding, adding a burn-in phase to mitigate missed detections near the beginning of causal and anticausal predictions, and making pseudo-label-based co-training aware of the non-causal model used for final inference. Results show the method reduces missed detections from over-tightening and improves both diarization accuracy and downstream multi-talker ASR performance. Accepted to IEEE SLT 2026.