Explainable Speech Emotion Recognition: Weighted Attribute Fairness to Model Demographic Contributions to Social Bias
New method reveals gender bias in HuBERT and WavLM models, quantifying how demographics influence emotion predictions.
A team of researchers has published a paper introducing a novel fairness modeling approach for Speech Emotion Recognition (SER) systems, titled 'Explainable Speech Emotion Recognition: Weighted Attribute Fairness to Model Demographic Contributions to Social Bias.' The work, led by Tomisin Ogunnubi, Yupei Li, and Björn Schuller, addresses a critical gap in AI ethics for audio processing. The authors argue that traditional fairness metrics like Equalised Odds and Demographic Parity are insufficient because they often overlook the complex, joint dependencies between multiple demographic attributes (like gender, age, accent) and a model's predictions. Their proposed method explicitly captures 'allocative bias' by learning these intricate relationships directly from model error patterns.
The researchers validated their new fairness metric on synthetic data before applying it to evaluate two prominent self-supervised learning (SSL) models: HuBERT and WavLM. These models were fine-tuned on the widely-used CREMA-D dataset for emotion recognition. The results were revealing. The new Weighted Attribute Fairness model successfully captured more mutual information between protected attributes and model biases compared to standard approaches. More importantly, it provided a quantifiable measure of the absolute contribution of individual demographic attributes to the overall bias, moving beyond simple binary fairness checks.
A key finding from the analysis, detailed in the 5-page paper, was the detection of indications of gender bias in both the HuBERT and WavLM models. This demonstrates the practical utility of the method for auditing real-world AI systems deployed in sensitive domains like mental health monitoring and education, where biased emotion predictions could cause significant harm. The work represents a step toward more explainable and accountable AI by providing developers with a tool to not just detect bias, but to understand its specific demographic sources.
- Proposes 'Weighted Attribute Fairness,' a new metric that models the joint dependency between demographics and model error, outperforming traditional fairness checks.
- Applied to HuBERT and WavLM models on CREMA-D data, it quantified individual attribute contributions to bias and revealed indications of gender bias.
- Provides an explainable framework for auditing SER systems in high-stakes domains like mental health, where biased predictions are harmful.
Why It Matters
As AI analyzes human emotion for healthcare and hiring, this tool helps developers pinpoint and fix demographic bias at its source.