New 'Speaker-Switch' Test Reveals AI Can Detect Genuine Conversation Dynamics

A new paper from researchers at arXiv (eess.AS 2606.02185) tackles a core challenge in conversational AI: how to tell if a model truly captures the give-and-take between two speakers rather than just memorizing each person's habits. The team—Nishchay Nilabh and Neeraj Kumar Sharma—introduces the Dyadic Distance Matrix (DDM), which encodes all pairwise similarities between every turn of two speakers across a full conversation. This matrix captures long-range cross-speaker dependencies, but the key question was whether the DDM reflects actual interaction or just individual speaker characteristics.

To answer that, they developed the speaker-switch test: replace one speaker's turns with those from an unrelated speaker in a different conversation, preserving turn-level statistics but destroying the original dyadic co-adaptation. Across four embedding types and classifiers, including ResNet-50 trained on the CANDOR corpus, real DDMs were consistently distinguishable from switched ones. Comparisons with LibriSpeech showed higher discriminability in naturalistic vs. read speech, highlighting the role of prosodic variability. GradCAM analysis revealed distinct structural signatures driving classification, confirming the test as a robust diagnostic for validating representations of dyadic interaction.