Practitioner Voices Summit: How Teachers Evaluate AI Tools through Deliberative Sensemaking
61 K-12 math educators developed personal rubrics, framing AI as assistants rather than coaches.
A new study from Stanford University researchers Dorottya Demszky, Christopher Mah, and Helen Higgins provides a crucial framework for how educators actually evaluate AI tools for classroom use. Their paper, 'Practitioner Voices Summit: How Teachers Evaluate AI Tools through Deliberative Sensemaking,' documents a two-day national summit involving 61 U.S. K-12 mathematics educators. The participants generated over 200 specific criteria for assessing AI tools, which were organized into four higher-order themes: Practical (ease of use, time-saving), Equitable (accessibility, fairness), Flexible (adaptability to different students), and Rigorous (academic quality, alignment with standards).
The research introduces the concept of 'deliberative sensemaking,' which integrates Technological Pedagogical Content Knowledge (TPACK) with teacher agency. This process revealed that teachers overwhelmingly view AI as an assistant to support student work rather than as a coaching tool for their own professional development. The study identified five key mechanisms that support effective evaluation: dedicated time for deliberation, artifact-centered discussions, collaborative reflection with diverse viewpoints, structured knowledge-building, and psychological safety. These findings highlight a critical gap where teachers are often excluded from adoption decisions despite being the primary users.
The implications are significant for multiple stakeholders. For edtech developers, the 200+ criteria offer a direct roadmap for creating tools teachers will actually adopt. For school leaders, the study provides a framework for inclusive decision-making processes. For professional learning designers, it outlines how to structure effective training around new technologies. The research demonstrates that when given proper structure and agency, teachers develop sophisticated, nuanced frameworks for technology evaluation that balance innovation with practical classroom realities.
- 61 K-12 math educators generated 200+ evaluation criteria across Practical, Equitable, Flexible, and Rigorous themes
- Teachers framed AI primarily as classroom assistants (87% of criteria) rather than professional coaching tools
- Five support mechanisms identified: deliberation time, artifact-centered discussion, collaborative reflection, knowledge-building, and psychological safety
Why It Matters
Provides a concrete framework for involving teachers in AI adoption decisions, directly impacting how $4B+ in edtech tools get evaluated and implemented.