StreamHear's semi-supervised ASR cuts labeling cost, boosts accuracy

Streaming automatic speech recognition (ASR) often crumbles when faced with domain-shifted audio—think financial calls or phone-quality dialogue—where labeled in-domain data is expensive and unlabeled audio is plentiful. A new arXiv paper from Zefang Liu, Chenyang Zhu, Sangwoo Cho, Xujun Peng, Shi-Xiong Zhang, and Sambit Sahu presents StreamHear, a semi-supervised pipeline that adapts a pretrained streaming model without needing extensive human annotations. The approach fine-tunes an offline transducer teacher on a small labeled training set, uses it to generate pseudo-labels on the unlabeled portion, and then fine-tunes the streaming student on the combined mixture.

To counter the noisy alignments that plague chunk-level streaming models, StreamHear introduces a prior-regularized dynamic-programming realignment step that anchors word boundaries using the ASR hypothesis itself. Tested across four datasets spanning financial calls, prepared read speech, and phone-quality dialogue, StreamHear consistently outperforms supervised student fine-tuning and significantly narrows the accuracy gap to the offline teacher. The work highlights a practical path for deploying streaming ASR in specialized domains, reducing dependence on costly labeled data while retaining low-latency inference.