How Fair is Software Fairness Testing?

A team of seven researchers, led by Ann Barcomb and Ronnie de Souza Santos, has published a provocative vision paper titled 'How Fair is Software Fairness Testing?' on arXiv. The paper fundamentally challenges the assumption that software fairness testing—a core method for evaluating AI systems—is a neutral, universally applicable science. Instead, the authors position it as a culturally situated practice, arguing that current approaches encode specific Western values while marginalizing others. They identify three critical dimensions of the problem: fairness metrics themselves reflect particular cultural biases, test datasets are overwhelmingly designed from Western contexts, and the entire process raises significant ethical concerns.

The paper details how current fairness testing excludes knowledge systems grounded in oral traditions, Indigenous languages, and non-digital communities, creating a significant blind spot. Furthermore, it highlights the ethical supply chain of AI, criticizing the reliance on low-paid data labeling labor in the Global South and the disproportionate environmental costs of large-scale model training on climate-vulnerable populations. The authors conclude that addressing these issues requires a paradigm shift—moving beyond universal metrics toward evaluation frameworks that actively respect cultural plurality. A key, radical proposal is acknowledging a community's 'right to refuse algorithmic mediation' altogether, suggesting fairness isn't just about better algorithms, but about democratic choice over their deployment.