New Research: AI Could Make Child Welfare Decisions Auditable

Most conversations about AI accountability go one direction: how do we make the technology transparent, explainable, and fair? A new paper from researcher Soheil Human flips it around. Instead of asking how to fix AI, he asks how digital tools could make our human institutions — courts, agencies, bureaucracies — more accountable to the people they serve. He calls this idea 'accountable digitality.'

The chosen test case is deliberately difficult: child-welfare judgments. These are the decisions about whether a child stays with their family, gets removed, or is placed elsewhere. Three things make them uniquely hard. First, the stakes are enormous — a child's safety and future. Second, almost everything is confidential, so you can't just publish the records. Third, judges must stay independent; you can't have software dictating rulings. At the same time, agencies genuinely need to learn from patterns — are certain families treated differently? Are certain reasons for removal overused?

The proposal is what the author calls 'needs-aware information systems.' In plain terms: software designed around what people actually need, rather than what's easiest to measure. It would let an agency see aggregate trends — like whether outcomes differ by neighborhood or income — without exposing any single family's private file. The paper describes a design logic and safeguards, not a finished tool.

The honest caveat is important. This is a conceptual paper. There's no data, no pilot program, no measured results. The author says the child-welfare case is used to illustrate and test the design thinking, not to prove it works. So treat this as a thoughtful blueprint, not a breakthrough. Still, the underlying question is worth asking: as more of government runs on software, who audits the humans?