Activation Steering for Accent Adaptation in Speech Foundation Models
New technique modifies speech model activations during inference, reducing word error rates across eight accents.
A research team led by Jinuo Sun and Yang Xiao has published a novel paper titled 'Activation Steering for Accent Adaptation in Speech Foundation Models' on arXiv. The work addresses a persistent challenge in automatic speech recognition (ASR): accent variability remains a major source of errors. Instead of traditional parameter fine-tuning, the researchers treat accent variation as an interpretable subspace within the model's hidden representations. They extracted layer-wise encoder activations and estimated specific 'mean-shift directions' that capture how representations change between accented and standard speech.
Through systematic analysis, the team discovered that accent information concentrates in a surprisingly narrow band of middle encoder layers, creating a clear 'accent sensitivity profile.' This finding enabled their key innovation: parameter-free accent steering. During inference, the method injects these pre-calculated directional adjustments directly into the model's activations, effectively guiding the representation toward a more standard pronunciation without modifying any underlying weights. Experiments demonstrated consistent word error rate reductions across eight different accents, proving the technique's effectiveness and efficiency compared to weight-based adaptation methods.
- Identifies accent information concentrated in specific middle layers of speech model encoders
- Enables parameter-free adaptation by steering activations during inference, avoiding costly fine-tuning
- Achieved measurable word error rate reductions across eight tested accents in experiments
Why It Matters
Enables more accurate, globally accessible speech recognition without retraining models, reducing bias and deployment costs.