Your Voice Quietly Reveals Your Gender and Nationality to AI

Researchers are trying to make speaker recognition neural networks less opaque. Speaker recognition networks learn latent representations called speaker embeddings from input utterances to recognise speaker identities, but their internal mechanisms remain largely opaque. To explain how these embeddings are organised, the authors apply a hierarchical clustering algorithm, Single-Linkage Clustering, to analyse whether some speaker embeddings naturally form clusters with hierarchical relationships, and evaluate the resulting organisation with Cluster-Class Matching. They also propose a new method, Hierarchical Cluster-Class Matching, to identify which hierarchical clusters best match individual semantic classes such as male and conjunctive semantic classes such as UK & male, quantifying the matching degree with a new metric called the L-score that makes imperfect matches diagnosable. Their results show the hierarchical clusters are interpreted using different classes related to speaker identity, gender, and nationality.