Investigating Group Relative Policy Optimization for Diffusion Transformer based Text-to-Audio Generation

A research team led by Yi Gu, Yanqing Liu, Chen Yang, and Sheng Zhao has published a paper investigating Group Relative Policy Optimization (GRPO) for improving Diffusion Transformer-based text-to-audio generation. The work addresses persistent challenges in accurately rendering complex audio prompts and achieving precise text-audio alignment, areas where previous methods using data augmentation and explicit timing conditioning have fallen short. The researchers' novel two-stage approach first leverages a large language model to generate high-fidelity, richly detailed audio captions, which substantially improves semantic alignment for ambiguous prompts. This enhanced dataset then serves as the foundation for the core innovation.

The team systematically applied the GRPO reinforcement learning algorithm to fine-tune their T2A model, experimenting with diverse reward functions including CLAP, KL, FAD, and their combinations to identify the key drivers of effective RL in audio synthesis. Their analysis reveals how specific reward designs impact final audio quality, with experimental results demonstrating that GRPO-based fine-tuning yields substantial gains in both synthesis fidelity and prompt adherence. This method represents a significant step forward in making AI-generated audio more reliable and contextually accurate, particularly for complex soundscapes described in text.