Self-EmoQ uses RL and psychology to give AI emotional speech

Current conversational AI lacks a mechanism to determine its own emotion before speaking, leading to flat or misaligned text-to-speech (TTS) synthesis. Self-EmoQ, accepted to ACL 2026 Findings and built by Yue Zhao and team, solves this with a novel emotion-planning framework. It works as a plug-and-play module initialized from pretrained LLMs and trained via reinforcement learning, where the 'actions' are emotions. A hybrid reward function combines imitation signals with theory-driven scoring based on Plutchik's wheel of emotions—a psychological model that defines eight primary emotions and their intensities. This allows the model to decide what emotion to express (e.g., joy, sadness, surprise) before generating the corresponding text, ensuring the downstream streaming TTS is emotionally grounded from the start.

Experiments on DailyDialog, EmoryNLP, IMEOCAP, and MELD show Self-EmoQ outperforms both prompting and finetuning baselines on emotion determination accuracy and response quality. The researchers also implemented a complete streaming pipeline for real-time deployment, with speech quality evaluations confirming excellent emotional alignment, contextual coherence, and expressive fluency. By treating emotion as a strategic decision rather than an afterthought, Self-EmoQ brings AI one step closer to human-like conversational nuance. Code and demos are available online.