Embedding-Based Intrusive Evaluation Metrics for Musical Source Separation Using MERT Representations
New metrics using MERT AI embeddings outperform traditional methods by better matching human perception of audio quality.
A new research paper from Paul Bereuter and Alois Sontacchi tackles a core problem in AI-powered audio processing: how to accurately and automatically evaluate the quality of musical source separation (MSS). MSS is the AI task of isolating individual components, like vocals or drums, from a mixed audio track. For years, the field has relied on BSS-Eval metrics, but recent studies show these metrics have a low correlation with how humans actually perceive audio quality, which is the true gold standard.
As an alternative, the researchers propose new "embedding-based intrusive" metrics. These leverage the powerful latent representations from a large self-supervised audio model called MERT. Specifically, they test two metrics: a mean squared error (MSE) and an intrusive variant of the Fréchet Audio Distance (FAD), both calculated on MERT embeddings. In experiments across two independent datasets, these new metrics demonstrated a stronger correlation with human perceptual ratings from listening tests than the traditional BSS-Eval metrics did, and this held true across different audio stems and separation model types.
This work, presented at the DAGA 2026 conference, represents a significant methodological shift. By using the rich, contextual understanding captured by models like MERT, the proposed metrics offer a more perceptually-aligned and reliable automated evaluation system. This advancement could accelerate the development of better separation models by providing researchers with a more accurate feedback loop, moving beyond flawed numerical scores to metrics that truly reflect what we hear.
- Proposes new embedding-based metrics using MERT AI model representations to evaluate audio separation quality.
- Metrics show stronger correlation with human listening test ratings than traditional BSS-Eval tools across multiple datasets.
- Provides a more reliable, automated evaluation method to accelerate development of better music source separation AI.
Why It Matters
Enables faster, more accurate development of AI tools for music production, remixing, and audio restoration by improving quality assessment.