Researchers unveil WordVoice for precise TTS control
New framework gives creators granular word-level control over AI voice synthesis...
Researchers from the South China University of Technology have developed WordVoice, a groundbreaking framework for LLM-based Text-to-Speech (TTS) systems that introduces explicit multi-dimensional word-level control. Published on arXiv, this work addresses a critical limitation in current TTS models where fine-grained stylistic control remains coarse and challenging to implement.
The team created WordVoice-5A, a massive 4.7k-hour bilingual dataset featuring five-dimensional word-level annotations (duration, boundary, energy, pitch and tone). They then built the WordVoice framework to transform implicit generation into an explicit, controllable process using a novel bound-token mechanism within the LLM. This enables adaptive multi-task prosodic planning and manual intervention. The system achieves superior decoupled control over multiple acoustic dimensions while maintaining competitive zero-shot synthesis stability.
- WordVoice introduces a bound-token mechanism for explicit acoustic planning in LLM-based TTS
- The WordVoice-5A dataset contains 4.7k hours of bilingual speech with 5D annotations (duration, boundary, energy, pitch, tone)
- Framework achieves precise word-level control while maintaining natural zero-shot synthesis quality
Why It Matters
This breakthrough enables professional creators to fine-tune AI voices with unprecedented precision for applications like audiobooks and dubbing.