Contextual Biasing for ASR in Speech LLM with Common Word Cues and Bias Word Position Prediction

IBM researchers have developed a novel approach to improve automatic speech recognition (ASR) in Speech Large Language Models (SLLMs) that addresses a critical weakness: accurately transcribing rare or specialized "bias words" not seen in training data. Traditional contextual biasing methods require users to provide phoneme representations via text prompts or specialized modules, which demands phonetic expertise or access to grapheme-to-phoneme (G2P) systems. The new method, detailed in arXiv paper 2604.12398, eliminates this barrier by using common words with partially similar pronunciations as acoustic cues instead.

This innovation enables end users to implement contextual biasing without any special phonetic knowledge or G2P tools. The system also incorporates bias word positional prediction through multi-output learning, enhancing robustness across different domains. In testing, the approach achieved a 16.3% reduction in bias word recognition errors compared to baseline systems, maintaining effectiveness even on out-of-domain data where traditional methods typically struggle.

The breakthrough represents a significant step toward more accessible and practical speech AI applications. By removing the technical barriers associated with phoneme-based approaches, IBM's method makes advanced contextual biasing available to a wider range of applications and users, from customer service systems to specialized domain transcription tools that need to handle industry-specific terminology with high accuracy.