Simulating Novice Students Using Machine Unlearning and Relearning in Large Language Models

A research team from Jiajia Song, Zhihan Guo, and Jionghao Lin has published a novel approach to creating more realistic AI-simulated students for educational applications. Their method addresses a key limitation in current learning-by-teaching systems: even when prompted to "act like a novice," large language models (LLMs) like GPT-4 or Claude often produce expert-level explanations, causing the simulated student to drift beyond intended knowledge levels. This undermines the credibility of the simulation for studying teaching processes.

The researchers propose using machine unlearning—a technique that selectively removes specific knowledge from trained models—to transform knowledgeable LLMs into stable novice-level agents. In their study using Python programming concepts, they applied unlearning to create teachable agents that maintain consistent novice behavior. These agents then demonstrated measurable knowledge recovery through structured learning-by-teaching dialogues, with analysis revealing identifiable trajectories of conceptual change and teaching effectiveness.

Results showed three significant findings: unlearning produced more novice-like responses than prompt-only baselines, agents recovered a measurable portion of unlearned knowledge under structured exposure, and dialogue analyses revealed predictable patterns of conceptual change. This approach enables more authentic educational simulations where human students can practice teaching skills without the AI "student" accidentally revealing expert knowledge that would undermine the learning experience.