SOMtime the World Ain$'$t Fair: Violating Fairness Using Self-Organizing Maps

A new research paper titled 'SOMtime the World Ain't Fair: Violating Fairness Using Self-Organizing Maps' reveals a fundamental vulnerability in current AI fairness approaches. Researchers Joseph Bingham, Netanel Arussy, and Dvir Aran demonstrate that sensitive attributes like age and income can be reconstructed from purely unsupervised embeddings, even when explicitly excluded from training data—directly contradicting the widely held assumption that unsupervised representations remain neutral when sensitive attributes are withheld.

The team's SOMtime method, based on high-capacity Self-Organizing Maps, achieved Spearman correlations up to 0.85 for recovering withheld sensitive attributes on real-world datasets including the World Values Survey across five countries and the Census-Income dataset. In stark contrast, traditional methods like PCA and UMAP typically remained below 0.23 correlation, with t-SNE and autoencoders reaching at most 0.34. The research shows that unsupervised segmentation of SOMtime embeddings produces demographically skewed clusters, demonstrating downstream fairness risks without any supervised task involvement.

This finding fundamentally challenges the 'fairness through unawareness' paradigm that underpins many current AI fairness approaches. The implications are significant: fairness auditing must now extend to unsupervised components of machine learning pipelines, not just supervised models. The researchers have made their code publicly available, enabling broader validation and development of mitigation strategies for this newly identified vulnerability in AI systems.