New AI model generates realistic listener nodding for avatars in real-time

In human conversation, subtle nonverbal cues like nodding signal active listening and encourage the speaker to continue. This paper from Kato et al. (Kyoto University) addresses the challenge of generating realistic listener nodding for conversational avatars. The proposed model consists of two modules: a timing prediction module and a kinematic parameter prediction module, each built on a dyadic attention network that tracks both speaker and listener voice activity. Unlike prior work that only predicts when to nod, this model also predicts the kinematic parameters (e.g., speed, amplitude) of the nodding motion, adapting to the specific dialogue context in real time.

The model is lightweight enough for real-time operation and has been integrated into a live avatar dialogue system. Subjective evaluation experiments show it significantly outperforms baselines that use either stochastic timing or fixed-motion nodding. The code and trained models are publicly available. This work enhances naturalness in human-avatar interaction, with applications in virtual assistants, telepresence robots, and digital humans. Future work may extend to other nonverbal cues like gaze and facial expressions.