New SRD metric exposes privacy leaks in voice anonymisation systems
Traditional EER metrics miss critical privacy flaws – SRD finds them.
Current voice anonymisation evaluation relies heavily on automatic speaker verification metrics like the equal error rate (EER). However, EER depends on the classifier and operating point, giving an incomplete or misleading picture of privacy risk. In a new preprint on arXiv (arXiv:2605.07291), researchers from multiple European institutions introduce similarity rank disclosure (SRD), a threshold-independent, information-theoretic metric that operates directly on feature representations rather than classifier decisions. This allows analysis of both average and worst-case disclosure.
Testing SRD on speaker embeddings, fundamental frequency, and phone embeddings from 2024 VoicePrivacy Challenge systems, the team found that SRD exposed privacy leaks and system-specific weaknesses completely missed by EER-based evaluation. The findings highlight the merit of representation-level metrics and position SRD as a flexible, interpretable tool for voice anonymisation evaluation. For developers of voice privacy systems, this new metric could become essential for reliable security testing.
- SRD is an information-theoretic metric that evaluates privacy at the feature representation level, not classifier outputs.
- Unlike EER, SRD is threshold-independent, enabling detection of both average and worst-case privacy leaks.
- Applied to 2024 VoicePrivacy Challenge systems, SRD revealed system-specific weaknesses that EER entirely missed.
Why It Matters
Provides a more reliable, interpretable benchmark for voice privacy systems used in real-world AI deployments.