A fresh preprint landed on arXiv this month, and it's quietly picking a fight with the entire voice AI industry. "From Metrics to Natural Dialogue: French Full-Duplex Benchmark" by Hamid Soltani and Gilles Boulianne of Luqia Technologies isn't just another leaderboard. It's an argument that we've been measuring the wrong thing all along. Full-duplex models—those that listen and speak at the same time—have been graded on turn-taking latency, interruption handling, and overlap statistics. The authors say that's not enough. Natural dialogue is the goal, not a stopwatch.
Here's why that matters. France has a full-duplex model of its own: Kyutai's Moshi, one of the first open models in this space, which has already spawned community fine-tunes like Muchi. Now France has a French-language benchmark. That's not a coincidence—it's a national stack. The English-centric suite Full-Duplex-Bench (FDB) v3, released in April 2026, has been the reference test. But if you're a French model, being graded on English conversations is like judging a croissant by how well it holds burger patties.
The winners are obvious: Kyutai, the French open-model ecosystem, and anyone betting on language sovereignty in AI. The losers? Cascaded, turn-based voice pipelines—the ASR → LLM → TTS assembly lines that power most commercial voice assistants. The full-duplex literature already treats them as the inferior architecture. If "natural dialogue" becomes the accepted frame, every cascade is suddenly measuring itself on the wrong axis. That's like training for a marathon and showing up to a dance contest.
But here's the twist: the paper's title is a quiet indictment of benchmarks in general. "From Metrics to Natural Dialogue" implies that the moment you define naturalness as a number, models optimize the number. Turn-taking stats are computable, comparable, and gameable. Naturalness is none of those. A supporting hint from adjacent research suggests current full-duplex models are trained with token-level likelihood maximization, which doesn't directly optimize conversational properties. So the critique is bigger than French. It's about the entire evaluation pipeline.
For the average person, this is the difference between a voice assistant that feels like a walkie-talkie and one that feels like a friend. You don't care about latency percentiles; you care about whether it interrupts you rudely or pauses awkwardly. The French benchmark is a step toward measuring that—but it's also a new way to fail. Any full-duplex model that doesn't handle French is now squeezed. And any company that hasn't thought about naturalness beyond metrics might be building for a game that's ending.
So what happens next? The paper could push the field toward more language-specific, naturalness-focused benchmarks. Trends suggest we might see a wave of "dialogue quality" suites that try to quantify the unquantifiable. Or the industry could ignore it and keep shipping cascades. My prediction: within a year, "full-duplex" will be a checkbox on every voice AI pitch deck, and "natural dialogue" will be the new battleground. The question is whether anyone can actually measure it—or if we'll just keep counting milliseconds while users keep hanging up.