FlowTTS-GRPO uses online RL to boost TTS speaker similarity and quality

FlowTTS-GRPO introduces an online reinforcement learning approach tailored for flow-matching (FM) based text-to-speech (TTS) systems, a category that has been under-explored compared to large language models (LLMs). The framework works by converting the ordinary differential equation (ODE) trajectories typical in FM into stochastic differential equation (SDE) paths, allowing direct fine-tuning of open-source FM models without needing auxiliary models or extra components.

The paper identifies three practical optimizations that significantly improve performance: (1) omitting classifier-free guidance (CFG) during training speeds up convergence; (2) synthesizing hard cases during training improves model robustness; (3) applying RL specifically to the FM component rather than all submodules enhances audio-detail metrics. When tested on CosyVoice 3.0 and F5-TTS, FlowTTS-GRPO achieved objective and subjective gains in speaker similarity and perceptual quality. Notably, F5-TTS also showed improved intelligibility. The authors also found that a weighted combination of rewards converges faster than a probabilistic scheme. This work, accepted at Interspeech 2026, demonstrates that online RL can effectively optimize flow-matching TTS models in a stable, sample-efficient manner.