New AI System Grades Students' Spoken Answers — With Teacher in the Loop
Could AI grade your spoken exams? A new system keeps teachers in charge.
A researcher built CALLIOPE, a system that connects spoken assessment responses to versioned instructional material, with learner-turn recording and transcription, adaptive questioning, two-provider rubric scoring, educator review and exportable evidence. In a final release rehearsal, three spoken fixtures — part of a set that also included a silence control, exercised across two release runs — received aggregate AI scores of 100, 62 and 8 out of 100, and two elicited provider-disagreement flags. All eight retrieved audio files were byte-identical to their inputs. The paper says these observations establish operation of specific exercised paths, not scoring validity or learning gains, and reports no human-participant outcomes.
- It scores spoken answers against the actual lesson material, not just whether they sound confident
- Three test answers scored 100, 62 and 8 out of 100 — and two raised flags that the two AI graders disagreed
- No real students took part, so we still don't know whether the grades are actually fair
Why It Matters
If AI starts grading spoken exams, expect a human teacher to stay in charge and flag uncertainty.