HPRO framework boosts emotional TTS with hierarchical reward optimization

Large Language Model (LLM)-based Text-to-Speech (TTS) models have achieved impressive naturalness, but standard supervised fine-tuning often results in monotonous prosody that lacks emotional expressiveness. While preference-driven optimization is a promising alternative, existing approaches suffer from two structural mismatches: information conflict, where content and emotion in a shared latent space produce conflicting gradients leading to reward hacking and semantic degradation; and scale gap, where sparse sentence-level rewards struggle to guide dense frame-level generation.

To address these challenges, the research team introduces HPRO (Hierarchical Progressive Reward Optimization). The framework features the HD-Emo codec, a novel differentiable reward model that separates speech into content and style preference tokens, structurally isolating emotional optimization from semantic content. This allows HPRO to bridge the scale gap by progressively aligning frame-, word-, and sentence-level objectives. Experiments demonstrate that HPRO significantly enhances emotional expressiveness while effectively preserving linguistic intelligibility. The code and audio samples are publicly available on GitHub.