New Study: Setting Your Own AI Rules Beats Any Fancy AI Tool

A researcher at a European university ran a seven-week experiment with 33 master's students learning data analytics. Half were given a special AI study coach built into their coding notebooks. It doled out hints one step at a time, refused to write code for them, capped how many hints they could get, and made them write a short reflection after each session. The other half were free to use ChatGPT, Copilot, or anything else they liked. The hope was that the guided group would learn more and lean on AI less.

That hope mostly didn't pan out. Assignment scores were basically identical in both groups. The guided group felt more confident but wasted their limited hints; the unrestricted group liked their tools but worried about how much they depended on them. Interestingly, students in both groups said the most valuable thing they took away was simply noticing how much they relied on AI in the first place.

What actually predicted success wasn't the tool at all — it was the student. Those who had thought through their own personal rules for when AI is and isn't allowed did better in both groups. Those who understood how these models actually work — every one of them self-taught, not from a class — used AI most deliberately and got the highest grades. In other words, good judgment beat good software.

The author argues schools are responding the wrong way. Banning AI or handing students a watered-down version doesn't build the skill that matters. Instead, she proposes grading the thinking behind AI-assisted work, and teaching AI literacy as a core subject like writing or math. Right now, she notes, that skill is picked up by luck, which means the people who need it most often miss it.