EE-Eval framework automates interactivity assessment for AI learning materials

Large language models now enable rapid generation of interactive learning materials, but evaluating their interaction quality remains a challenge. Existing benchmarks focus on code executability or visual fidelity, ignoring dynamic behaviors like learner-controlled state transitions and context-sensitive responses that shape conceptual understanding. To address this, researchers present EE-Eval (arXiv:2606.31012), an automated evaluation framework that formalizes interactivity as a finite space of learner-controllable states and transitions, represented as a Finite State Machine (FSM). By extracting FSMs from AI-generated explorable explanations, EE-Eval externalizes implicit interaction logic into an explicit, machine-interpretable graph, then compares each generated FSM to an ideal FSM encoding pedagogical intent using graph-based metrics and embedding-based comparisons of states, actions, and feedback.

Across thousands of generated explanations spanning 127 concepts from 6 AI models, EE-Eval consistently differentiates interaction quality beyond surface-level criteria like functional correctness or visual quality. It exhibits substantially stronger alignment with human judgments of interactivity and pedagogical effectiveness than existing baselines. By framing interactivity as testable behavioral models rather than an emergent byproduct of LLM generation, EE-Eval transforms evaluation into a reflective diagnostic tool. This enables pedagogically grounded human-AI collaboration in creating interactive educational content, offering a scalable way to audit and improve AI-generated learning experiences.