Researchers improve BEST-RQ speech model with 12% WER reduction

BEST-RQ is a simple self-supervised speech representation learning method that uses a fixed online quantizer to generate pseudo-labels. While efficient, this simplification provides weaker supervision compared to HuBERT-style models that iteratively refine pseudo-labels offline. Researchers from RWTH Aachen University have now proposed three modifications to improve online pseudo-label quality without sacrificing simplicity.

First, they replace the quantizer's linear projection with Principal Component Analysis (PCA) to better capture speech features. Second, they introduce iterative codebook refinement, updating the codebook entries during training for more representative pseudo-labels. Third, they add a second codebook updated via distillation from the first, further boosting label quality. On the LibriSpeech 960-hour pre-training set and 100-hour supervised fine-tuning, their model cut WER from 10.1% to 8.8% on the test-other set—a 12% relative improvement. This work was accepted at Interspeech 2026.