Audio & Speech

New AI Research Could Let You Interrupt Your Voice Assistant

New AI Research Could Let You Interrupt Your Voice Assistant

⚡Imagine talking to AI without awkward pauses—this could make conversations natural.

Deep Dive

Full-duplex speech models can listen and speak simultaneously, enabling natural interaction, but become increasingly difficult to control as the conversation history grows. When used as user simulators, this lack of control can cause them to deviate from prescribed scenarios and produce unreliable evaluation outcomes.

We introduce SimIF-Bench (Simulator Instruction-Following Benchmark), which evaluates whether a conversational model stays within a prescribed scenario and completes multiple goals in the required order. The benchmark reveals that current open-source full-duplex models struggle to follow such constraints.

We then introduce a Group Reward-Decoupled Normalization Policy Optimization (GDPO)-based training recipe that enables a full-duplex model to follow textual instructions during an ongoing conversation while maintaining its turn-taking ability. By connecting the resulting SteerablePlex to an asynchronous backend language model that monitors the conversation and provides instructions when needed, we build a more controllable full-duplex user simulator that follows multi-stage constraints more reliably than existing open-source models and GPT-Realtime.

Key Points
  • SteerablePlex is a research project that aims to make AI listen and speak at the same time, like humans.
  • This could let you interrupt voice assistants and have more natural conversations, saving time and frustration.
  • It's still early research, so you won't see it in Siri or Alexa anytime soon, and privacy concerns remain.

Why It Matters

This could make AI conversations feel human, saving you time and frustration in daily tasks.

📬 Get the top 10 AI stories daily