Xemo-Talker brings lifelike emotions to AI talking heads

Xemo-Talker is a new audio-driven talking portrait synthesis system designed for explicit emotion control. Its key insight: emotional cues are distributed across the motion space, but focusing discriminative supervision on less-principal components achieves a better balance between emotion expression and lip synchronization, since principal components mostly encode articulation and pose. The system first learns a neutral speech-to-motion mapping, then adds a lightweight emotion branch guided by less-principal subspace supervision. A Tri-Loss combines inter-class separation, intra-class compactness, and less-principal contrastive learning. Given audio, a reference image, and an emotion label, Xemo-Talker achieves state-of-the-art emotion classification accuracy while maintaining competitive lip synchronization and high inference efficiency. The source code is publicly available, according to the paper.