AI That Explains with Your Hobbies: Memorable but Flawed

Imagine you're learning a tricky concept in computer science, like how a program loops through data. Instead of a dry textbook explanation, you ask ChatGPT to explain it using something you love, say, baking pizza. The AI compares the process to making dough and adding toppings. It feels fun and easy to remember. But what if the comparison is actually wrong in a subtle way? A new study found this happens more often than you'd think.

The study, presented at a computing education conference, gave ten college students explanations coded with analogies tailored to their interests. Students overwhelmingly said these personalized explanations were more engaging and memorable than plain technical ones. But here's the twist: when it came to trusting the analogy, students split. Some trusted it more because it felt personal. Others became suspicious, wondering why the AI was catering to their hobbies in the first place.

The most important finding? Only students who already knew a lot about the hobby — the thing the analogy was comparing to — could spot when the comparison didn't quite work. A pizza-making analogy might be structurally flawed in a way that only a real baker would notice. Someone learning programming, who is already struggling with the new concept, usually lacks that deeper knowledge to catch the error. That puts them in a tough spot: they remember a flawed explanation, and it quietly teaches them something wrong.

The researchers call this "two-sided analogy auditing." When you hear an analogy, on the abstract concept you're still a student. But on the familiar topic, you're actually the expert. AI tools should recognize this and adapt. Instead of only asking what you're interested in, they should ask what you already understand. A flawed analogy, the study suggests, shouldn't be dismissed. It can be a teaching moment — if the student is in a position to inspect it.