Fairness Audits of Institutional Risk Models in Deployed ML Pipelines

A new fairness audit of a deployed machine learning pipeline at Centennial College reveals systematic misallocation of support resources. The study, published on arXiv by Kelly McConvey and colleagues, replicates an Early Warning System (EWS) used to identify at-risk students. Using institutional training data and design specifications, the researchers evaluated disparities by gender, age, and residency status across the full pipeline—from training data to model predictions to post-processing. Standard fairness metrics showed that younger, male, and international students are disproportionately flagged for support, even when many ultimately succeed, while older and female students with comparable dropout risk are under-identified. Post-processing amplifies these disparities by collapsing heterogeneous probabilities into percentile-based risk tiers.

The audit builds on prior ethnographic work that introduced the ASP-HEI Cycle, a framework for understanding algorithmic systems in higher education. The methodology is designed to be replicable, offering a template for auditing other institutional ML systems. The findings highlight the importance of evaluating construct validity alongside statistical fairness, as disparities can emerge and compound across different stages of the pipeline. This work contributes to a broader research program investigating algorithms, student data, and power in higher education, underscoring the need for transparency and accountability in deployed ML systems.