TurnNat measures turn-taking naturalness in spoken dialogue automatically

Turn-taking is essential for natural full-duplex spoken dialogue systems (e.g., voice assistants, conversational AI), yet automatic evaluation has been limited to human judgments or simplistic timing metrics. Researchers introduce TurnNat, a likelihood-based framework that measures how natural the timing of speaker turns is in two-channel audio. The system trains a causal turn-taking prediction model on thousands of natural dyadic conversations to learn typical voice-activity transition patterns. At inference, it computes the negative log-likelihood (NLL) of observed future voice-activity states — lower NLL means more natural timing.

TurnNat breaks dialogue into turn-taking boundary units (TBUs) around utterance onsets and offsets, and scores each TBU. The final dialogue-level naturalness score aggregates mean and tail TBU scores, capturing both typical and rare timing failures. To validate, the authors created a benchmark with paired natural and artificially perturbed clips (e.g., inserted delays, early cutoffs). Human raters confirmed the perturbations were unnatural, and TurnNat accurately distinguished them across various failure types — outperforming existing metrics by a wide margin. This work, presented as a preprint on arXiv (2607.01345), provides a scalable and consistent alternative to costly human evaluation, enabling faster iteration in voice AI development.