MMedFD benchmark brings full-duplex ASR to real-world healthcare

MMedFD tackles a critical gap in clinical speech recognition: handling real-time, overlapping conversations where patients and doctors speak simultaneously. Unlike most ASR benchmarks that assume clean, turn-based audio, this dataset captures full-duplex interactions from a live AI assistant in Chinese hospitals. It contains 5,805 annotated sessions with synchronized user and mixed-channel views, plus RTTM/CTM timing files and role labels. The team developed a model-agnostic pipeline for streaming segmentation, speaker attribution, and dialogue memory, then fine-tuned Whisper-small on role-concatenated audio for long-context recognition.

Evaluation goes beyond standard WER and CER with a new metric, HC-WER, which measures concept-level accuracy specifically for healthcare terminology. LLM-generated responses were assessed using rubric-based and pairwise protocols. MMedFD is publicly available and provides a reproducible benchmark for streaming ASR and end-to-end duplex agents. This work is especially timely as voice-based AI assistants become more common in clinical settings, where latency and speaker overlap are unavoidable.