New AI Can Make Voices Laugh, Sigh, and Gasp Like Real People
The robotic voice on your phone is about to sound a lot more human.
Think about the last time you heard a computer voice read something out loud. It probably got the words right but sounded slightly dead. That's because real speech is full of extra noises — a small laugh, a sigh, a sharp intake of breath, a hesitant "um." These are called nonverbal vocalizations, or NVVs for short. They carry a huge amount of feeling, and most AI voice systems simply leave them out.
A research team fixed that. They built a system that treats these human noises as their own building blocks, giving each of 16 types — laughs, sighs, gasps and more — its own dedicated marker inside the AI. That lets the software decide when to drop one in mid-sentence instead of just at the end. They also trained on a mix of Mandarin and English speech, then deliberately boosted the rarer noises so the AI wouldn't ignore them. The result: first place overall in a 2026 speech challenge, and first in Mandarin.
Why should you care? Because the voice you hear from a customer service line, a navigation app, an audiobook, or a video game is only as good as its emotional range. Adding a nervous laugh or a tired sigh is the difference between a voice that reads to you and a voice that performs for you. For anyone dubbing movies into other languages, this matters even more — dubbing often loses the emotion entirely.
The catch is honesty. The same technology that makes a friendly narrator also makes it much harder to tell a real person from a fake one over the phone. And the team had to carefully balance three things at once: getting the noises right, keeping the actual words clear, and making sure the whole thing still sounds good. Nailing all three is genuinely hard, which is why this problem stayed unsolved for so long.
- The AI handles 16 different human noises — laughs, sighs, gasps and more — and knows when to slip them into the middle of a sentence, not just at the end.
- It was trained on both Mandarin and English, and took first place overall in a 2026 international speech challenge, plus first in Mandarin.
- The team boosted the rarest noises during training, because AI tends to ignore whatever it sees least often — the same reason voice assistants rarely sound surprised.
Why It Matters
More convincing fake voices will improve audiobooks and dubbing, but also make phone scams harder to spot.