A Representation-Level Assessment of Bias Mitigation in Foundation Models

A team of researchers from IBM and University College Dublin has published a new study, 'A Representation-Level Assessment of Bias Mitigation in Foundation Models,' accepted at ECML-PKDD 2025. The research investigates how successful bias mitigation techniques fundamentally reshape the internal embedding spaces of popular foundation models. Using BERT (an encoder-only model) and Llama2 (a decoder-only model) as representative architectures, the team compared baseline versions with their bias-mitigated variants. They specifically measured shifts in the geometric associations between gender and occupation terms within the models' high-dimensional vector spaces.

The findings reveal that effective bias mitigation directly reduces gender-occupation disparities in these embeddings, leading to more neutral and balanced internal representations. These representational shifts were consistent across both model types, suggesting fairness improvements manifest as interpretable, geometric transformations. This provides a new internal audit mechanism for model behavior. To further promote assessment of decoder models like Llama2, the team introduced and publicly released WinoDec, a novel dataset of 4,000 sequences containing gender and occupation terms. The work establishes embedding space analysis as a valuable, interpretable tool for validating the effectiveness of debiasing methods, moving beyond just output-based fairness metrics.