New AI Speech Library Covers 28 Languages — Including Voice Cloning

A team of researchers has released CVSS-X, a free collection of computer-generated speech designed to teach AI how to translate spoken language. It contains roughly 240,000 paired audio clips for each of 28 languages, adding up to more than 16,000 hours of speech. That makes it about eight times larger than the earlier version, and it now works in the opposite direction: English going out into 28 languages across 12 language families, rather than 21 languages coming into English.

Why should you care about a pile of audio files? Because data is the fuel for translation tools. Every time you use earbuds that translate a conversation in real time, or watch a video dubbed into English, an AI model is doing the work — and it got good by studying millions of examples like these. Datasets covering many languages, especially ones big tech usually ignores, are what make those features available to more people, in more places, at lower cost.

The collection comes in two flavours. One uses a couple of standard voices per language, which is what researchers typically test with. The other uses voice cloning, meaning a speaker's own voice can carry over into another language — imagine a video call where you speak Portuguese but your colleague hears you in English, in your voice. That's the part that feels like magic, and also the part that raises obvious questions about impersonation.

The honest catch: every clip is AI-generated, not recorded from real people. So the models trained on it may handle textbook sentences well but stumble on thick accents, slang, sarcasm or emotional tone. The license also allows research use only, so companies can't just drop it into a product without a separate deal. And nothing here is something you can download and use today — it's an ingredient, not a finished app.