Frozen SSL beats fine-tuned models for cross-corpus audio MOS prediction

Automatic Mean Opinion Score (MOS) prediction is critical for evaluating synthetic speech and audio enhancement systems, but models often fail when encountering data from unseen domains. To address this, Mustafa Ozan Duman and Ahmet Emir Dirik conducted a comprehensive benchmarking of three architectural frameworks: Frozen Self-Supervised Learning (SSL-FRZ), Fine-Tuned SSL (SSL-FT), and a Video Vision Transformer (ViViT). The study used a consolidated corpus of 130,000 samples across 19 diverse datasets, followed by a purified English-only corpus of 17 datasets. A systematic Leave-One-Dataset-Out (LODO) protocol measured the generalization gap between seen and unseen distributions, providing a rigorous test of domain robustness.

The results showed that an English-only purified corpus consistently improved predictive precision across all architectures. While SSL-FT achieved the highest performance on seen validation data, SSL-FRZ demonstrated superior robustness on unseen distributions, posting a competitive Mean Squared Error (MSE) of 0.36 on the URGENT 2024 benchmark—only slightly behind domain-optimized SOTA metrics (MSE 0.30). The ViViT architecture remained stable in English-only trials but fell short of SSL-based models in total capacity. LODO validation confirmed that frozen SSL embeddings combined with deep Transformer encoders offer the most stable and scalable solution for universal speech quality assessment. To support further research, the top-performing English-only SSL-Transformer model and weights are publicly available on Hugging Face.