Game-Time: Evaluating Temporal Dynamics in Spoken Language Models

Conversational spoken language models (SLMs) promise real-time speech interaction, but a critical blind spot has emerged: their ability to manage timing, tempo, and simultaneous speaking—collectively called temporal dynamics. To address this, researchers from MIT, National Taiwan University, and Academia Sinica introduced the Game-Time Benchmark, a systematic evaluation framework inspired by how humans learn language through activities. The benchmark includes basic instruction-following tasks and advanced scenarios with temporal constraints, such as adhering to a specific pace or delivering synchronized responses. The team tested a range of diverse SLM architectures and found a striking performance gap: while state-of-the-art models performed well on basic tasks, many contemporary systems still struggled with even fundamental instruction-following. More alarmingly, nearly every model saw substantial degradation under temporal constraints, highlighting persistent weaknesses in time awareness and full-duplex interaction—the ability to both listen and speak simultaneously.

The results underscore that current SLMs are far from achieving the fluid, human-like conversational dynamics needed for applications like real-time assistants, voice-controlled devices, and interactive AI characters. The Game-Time Benchmark, accepted to the prestigious ICASSP 2026 conference, provides researchers with a standardized way to measure and improve these temporal capabilities. The accompanying demos and datasets are publicly available, aiming to guide future research toward more temporally-aware conversational AI. For developers building voice-based products, the findings signal a critical need to go beyond basic speech understanding and focus on the nuanced timing that makes conversations feel natural.