Colleges' AI Student-Risk Alerts Look Fair, But Mostly Aren't

Colleges increasingly buy AI tools called Early Warning Systems (software that guesses which students are at risk of failing or dropping out). The problem: these systems are built by outside vendors, and schools can't open them up, inspect them, or change how they work. All a college can do is nudge the answers the system spits out.

A team of researchers wanted to know what that actually accomplishes. They used real student records from a public college in Ontario, Canada, and tested six different fairness fixes on a research version of an early warning system. Each fix was designed to make the AI treat students more equally across groups — by race, income, gender, and so on.

The results were uncomfortable. The fixes didn't consistently reduce unfairness. They mostly moved it around. A tool might flag fewer students from one group but more from another, or trade one kind of mistake for a different one. Two of the six fixes actually ended up favoring students who were already advantaged, because they defined 'disadvantaged' by group size — so small groups got overlooked. Small, marginalized groups of students remained poorly served in every scenario the team tested.

The authors coined a name for this: 'fairness theatre.' The numbers on the vendor's dashboard start looking equal, meetings feel productive, and everyone can say the right things. Meanwhile, the actual burdens fall unevenly, and for some students the situation gets worse, not better. The real lesson isn't that fairness fixes are useless — it's that when a school can't shape the underlying system, its ability to make that system fair is severely limited. If your college or your child's school uses one of these tools, the fair-looking dashboard may not mean much.