Social Workers Now Get to Design the AI That Helps Them

Most workers get AI tools handed to them with no say in how they work. This study flips that. Researchers ran eight workshops with 19 school social workers, letting them decide what "good" AI support looks like in their daily work. The workers chose their own goals — and one big goal was having AI push them to notice their own assumptions and biases. Then they built a test to measure it.

This matters because AI is moving into every workplace, from hospitals to offices to schools. Right now, tech companies usually decide what an AI should do and whether it works. Workers just have to accept it. This project shows a different path: front-line workers can define meaningful success for themselves. The social workers used their professional experience to create a scoring guide — a rubric — that tells an AI how to judge another AI's responses.

The results were encouraging. The worker-designed test could actually tell the difference between six leading AI models, and the workers' own ratings closely matched the AI judge's ratings. That means professional expertise can be turned into a reliable measuring tool. It's proof that "AI is good" doesn't have to be decided by engineers alone.

There's a catch though: this was one small group at one organization. The approach needs to be tried in other jobs before we know it works everywhere. And using AI to grade AI is still a young idea. But this study gives a real example of how to put the human — the worker — back in charge of the machines that are supposed to help them.