Study: LLM tutor timing, not prompt count, predicts math learning outcomes

A new study from researchers Rania Abdelghani, Peter Kaiser, and Kou Murayama, posted on arXiv, analyzed how 112 Grade-9 students interacted with a general-purpose LLM tutor during a math modeling task. 97 students completed both pre- and post-tests without AI. The researchers coded student turns for self-regulated learning functions, help-seeking content, and mathematical modeling activity—three dimensions they hypothesize capture epistemically proactive AI use.

The key finding: static summaries of AI use (e.g., total prompts, question types, behavioral diversity) did not predict post-test performance after controlling for prior knowledge. What did matter was the temporal trajectory. Students who shifted from early phases of verification and answer-seeking to later phases of conceptual or procedural help-seeking, combined with active mathematical work, showed significantly better learning outcomes. The study implies that AI tutors should be designed to guide students toward this temporal trajectory of epistemic proactivity, rather than simply maximizing interaction volume.