Study tests 5 speech encoders on 22 Indic languages for deepfake detection

A new arXiv paper from Varun Rai and colleagues examines whether pre-trained speech encoders can reliably detect spontaneous speech and synthetic (deepfake) audio in low-resource Indic languages. The team benchmarked five frozen transformer encoders—AST, Vaani-FastConformer, Wav2vec2, Whisper, and BEATs—across 22 Indic languages, going beyond the usual English-centric evaluations. They also tested generalization to out-of-domain synthetic speech using four different TTS systems.

Beyond accuracy metrics, the study introduces language isolation probing and centroid proximity analysis. Probing reveals an encoder-dependent trade-off: models that separate languages better tend to perform worse at spontaneity detection, and vice versa. More critically, centroid analysis shows that a model's ability to generalize to unseen TTS-generated speech is predicted by how close the training TTS embeddings are to the test synthetic embeddings—not by how far the model is from natural speech. This counters common assumptions and suggests that deepfake detector training data should be selected based on TTS system diversity and embedding coverage, not just the inclusion of natural speech examples.