EchoDistill boosts audio LLM accuracy by 4.18% under heavy noise
New self-distillation method tackles hallucinations in real-world noisy audio.
Get AI news that actually matters
One email a day. Zero fluff. Join 10,000+ professionals.
Audio Large Language Models (ALLMs) perform poorly in noisy environments, suffering from semantic drift and hallucinations. A team of 12 researchers introduces EchoDistill, an alignment-based noisy-to-clean self-distillation framework that tackles this without extra inference compute. The approach freezes a clean-audio teacher model to provide semantic references for a noisy-audio student at test time. The student samples candidate responses under noisy conditions, and group-relative policy optimization (GRPO) aligns them with the teacher's token-level consistency as a reward. This encourages acoustically grounded reasoning.
Experiments show EchoDistill significantly improves reliability. On the strongest baseline, it yields a 4.18% gain in GSR under heavy noise. When applied to Qwen-Omni, EchoDistill beats a GRPO-only variant by 3.02% in accuracy, 3.89% in noisy conditions, and 4.53% in GSR on average. The method requires no changes to the model architecture and introduces zero additional inference overhead, making it practical for production deployment in voice assistants, transcription, and hearing aids.
- EchoDistill uses a frozen clean-audio teacher to guide a noisy-audio student via GRPO reward shaping.
- Achieves 4.18% higher GSR (general semantic reliability) than state-of-the-art baselines under strong noise.
- On Qwen-Omni, beats GRPO-only by up to 4.53% in GSR with zero additional inference cost.
Why It Matters
Makes audio AI reliable in real-world noise, enabling robust voice assistants, transcription, and hearing aids.