IRAF: New adaptive filter lets voice AI ignore background noise in real-time

Researchers from multiple institutions, led by Tao Zhong, have introduced IRAF (Interference-Resilient Adaptive Fusion), a lightweight module designed to make full-duplex spoken dialogue systems robust to real-world noise. Full-duplex models allow voice agents to listen and speak simultaneously, but they struggle when interfering speakers leak into the user’s microphone. This corrupts the LLM’s conditioning, leading to unstable turn-taking and degraded responses. IRAF solves this by modulating the contribution of user audio to the LLM on a frame-by-frame basis. It uses a scalar reliability gate computed from embeddings of both target-speaker and user audio, then rescales user representations before fusing them with agent embeddings.

Experiments on MS-MARCO and InstructS2S-200K show consistent gains in response quality and full-duplex interaction under challenging acoustic conditions. The module is designed for streaming, meaning it can operate in real-time without significant latency. This work addresses a critical bottleneck for voice-first AI assistants deployed in cafes, open offices, or smart speakers where background chatter is common. By making the model selectively trust user audio, IRAF enables more natural, interruption-resistant conversations without requiring complex multi-microphone arrays.