StreamVoiceAnon+: Emotion-Preserving Streaming Speaker Anonymization via Frame-Level Acoustic Distillation

A team of researchers has introduced StreamVoiceAnon+, a novel AI model designed to anonymize a speaker's voice in real-time while crucially preserving the emotional content of their speech. The core challenge they address is that standard neural audio codec models, when used for speaker anonymization (SA), tend to strip away paralinguistic cues like emotion, defaulting to generic acoustic patterns. Their solution involves a two-part fine-tuning approach: supervised training on neutral-emotion utterance pairs from the same speaker, combined with a novel frame-level emotion distillation technique applied to the model's acoustic token hidden states. This allows the system to learn and retain the subtle acoustic features that convey emotion, separate from speaker identity.

The technical results are significant. Evaluated on the VoicePrivacy 2024 benchmark, StreamVoiceAnon+ achieves a 49.2% Unweighted Average Recall (UAR) for emotion preservation, marking a substantial 24% relative improvement over the baseline model's 39.7%. It also outperforms an emotion-prompt variant by 10%. Crucially, it maintains strong privacy protection with an Equal Error Rate (EER) of 49.0% and keeps speech highly intelligible with a Word Error Rate (WER) of just 5.77%. All modifications are confined to a fine-tuning stage that takes under two hours on four GPUs and adds zero latency during inference, enabling real-time streaming with a competitive 180ms delay. The model's code and demos are publicly available, providing a practical tool for privacy-sensitive applications like telehealth and customer service where emotional nuance is critical.