Study: Concrete Examples in AI Tutoring Cut Confusion, Long Replies Hurt Learning

A new study from University of Delaware researchers (Sharmin et al.) analyzed 16,851 conversational interactions between students and an LLM tutor in the StudyChat dataset, tracking how micro-level feedback features affected immediate learning outcomes. They focused on three features: concrete elaboration (analogies, comparison-based explanations, or worked examples), affective language (encouragement, empathy, apology), and response length. Using chi-square tests and Generalized Estimating Equations (GEE) on 1,718 cases where students expressed confusion and continued to a subsequent interaction, the authors found that concrete elaboration was associated with higher understanding and lower re-confusion in the next turn.

Surprisingly, empathetic language showed no significant association with either outcome, and longer response lengths were independently associated with lower understanding. These results challenge the common assumption that more supportive or verbose AI feedback is always better. Instead, they highlight the value of examining feedback across consecutive interactions rather than in isolation. The findings suggest that AI tutoring systems should be designed to give concise, example-rich explanations—and that empathy may be less impactful than the substance of the explanation itself for immediate learning gains.