PHONOS: PHOnetic Neutralization for Online Streaming Applications
New streaming AI module reduces non-native accent confidence by 81% with under 241ms latency.
A team of researchers has introduced PHONOS (PHOnetic Neutralization for Online Streaming Applications), a novel AI system designed to enhance speaker privacy by neutralizing non-native accents in real-time. The core innovation addresses a critical flaw in current speaker anonymization (SA) systems, which modify a voice's timbre but leave distinctive regional or foreign accents intact. These accents can act as a unique identifier, drastically narrowing the so-called 'anonymity set' and compromising privacy. PHONOS tackles this by pre-generating 'golden' reference utterances that preserve the source speaker's vocal timbre and speech rhythm but replace foreign phonetic segments (segmentals) with native equivalents.
The system operates with impressively low latency, making it suitable for live streaming applications. It employs a causal accent translator trained with joint cross-entropy and CTC losses, which maps non-native content tokens to native ones with a look-ahead of at most 40ms. Evaluations show PHONOS achieves an 81% reduction in non-native accent confidence, a result corroborated by human listening tests. Furthermore, the processed utterances move away from the original speaker in voice embedding space, reducing speaker linkability. The entire pipeline runs with a total latency under 241 milliseconds on a single GPU, demonstrating its feasibility for real-time communication platforms like video calls or live broadcasts where both voice quality and privacy are paramount.
- Neutralizes non-native accents to sound native-like, achieving an 81% reduction in accent confidence according to AI classifiers.
- Operates in real-time with a total system latency under 241ms and a processing look-ahead of just 40ms, enabling live streaming use.
- Uses a novel pipeline combining silence-aware DTW alignment, zero-shot voice conversion, and a causal translator to preserve speaker timbre while changing accent.
Why It Matters
This technology could fundamentally enhance privacy in global voice communications, protecting users from identification by their accent during sensitive calls or broadcasts.