Does Fine-tuning by Reinforcement Learning Improve Generalization in Binary Speech Deepfake Detection?

A team of researchers including Xin Wang has released a new paper on arXiv, investigating a novel method to improve AI's ability to detect speech deepfakes. The core challenge in this field is creating models that generalize to new, unseen audio forgeries, not just the ones they were trained on. While the standard approach involves pre-training a foundation model and then fine-tuning it with supervised learning (SFT), this paper draws inspiration from large language models (LLMs) and explores using reinforcement learning (RL) for the fine-tuning stage. Specifically, they applied a method called Group Relative Policy Optimization (GRPO) to speech detection models.

The technical experiments compared pure GRPO-based fine-tuning against SFT-only and hybrid setups across multiple detectors and test sets. The key finding is that models fine-tuned solely with GRPO showed improved performance on out-of-domain data—meaning they were better at spotting novel deepfake attacks—while still performing well on the target-domain data they were originally tuned for. This approach outperformed the other methods. The researchers' ablation studies suggest that the 'negative reward' mechanism within GRPO, which penalizes the model for incorrect classifications, may be a critical factor driving this improved generalization. This work, submitted to Interspeech 2026, points toward RL fine-tuning as a promising technique for building more robust and adaptable AI security tools in the arms race against synthetic media.