Khan Academy's Khanmigo paper reveals how to improve AI tutoring quality

Khan Academy, the nonprofit education platform, released a research paper titled "Methodologies for Improving the Quality of AI Tutoring in K-12 Education," accepted at the 27th International Conference on Artificial Intelligence in Education (AIED 2026). Authored by Tushar Udeshi and six colleagues, the paper draws on real-world experience from Khanmigo, Khan Academy's AI-powered tutor launched in 2023. The researchers argue that because LLMs are opaque, robust evaluation and live experimentation are critical for every change. They detail the specific metrics they use to assess tutoring quality and student engagement, and they share the experiments that led to measurable improvements.

Key improvements came from adjustments across four areas: model selection, prompt engineering, personalization, and AI agents. The paper highlights which changes moved the metrics, offering concrete evidence for what works in AI tutoring. For example, the team tested different LLMs and prompting strategies to reduce hallucinations and improve pedagogical responses. By iterating on personalization and agent-based features, they boosted engagement and learning outcomes. The paper is published in the Springer LNCS volume (vol. 16582) and is available on arXiv under DOI 10.48550/arXiv.2608.11259. This work provides a rare behind-the-scenes look at how a major education platform systematically improves an AI tutor, making it a valuable reference for both researchers and edtech practitioners.