New AI Makes Animated Faces Mouth Words Like Real People

When a video game character or a dubbed movie star speaks, the voice is often perfect but the mouth is not. Lips that slide, stretch or lag behind the audio make a face feel fake almost instantly. A team of researchers has now published a new method, accepted at the NeurIPS 2026 conference, that tackles this exact problem by focusing on how mouths actually move.

Their trick is to stop treating the whole face as one giant math problem. Instead, they boil visible mouth movement down to three motions: spreading (lips widening, like a smile), opening (the jaw dropping), and protrusion (lips pushing forward, like a kiss). A memory system learns which sounds tend to produce which motions, then a second step stitches those movements back onto the 3D face so the skin and shape stay consistent — no crumpling, tearing or weird stretching.

They tested it on two standard research datasets of talking faces. The AI scored better than previous methods at matching mouth movement and speed, and in a user study, real people preferred its lip sync and realism. That last part matters: human eyes are unforgiving, and a preference vote from viewers is a stronger signal than a number on a chart.

The catch: this is still a research result, not a product you can download, and it works from speech recorded first, then used to animate an existing 3D character — not live conversation. It was trained on limited datasets, so accents, singing and unusual mouth shapes may still trip it up. It also fixes mouth mechanics, not emotion: an angry line delivered with flawless lip sync can still look flat. The near-term winners are dubbing, games and virtual avatars.