Researchers build AI autograders from solutions to cut grading time

Researchers Geoffrey Challen and Ben Nordick from the University at Buffalo have developed Questioner, a novel autograding system that accelerates the creation of programming assignments by generating test cases automatically from provided solutions.

Published in the SIGCSE TS 2025 proceedings, their paper demonstrates how this approach eliminates the tedious manual enumeration of test cases, which often produces inaccurate graders. Questioner has been used to author nearly 800 programming questions for Java and Kotlin, supporting thousands of students in evaluating millions of submissions over four years in a large CS1 course. The method leverages the fact that question authors can supply a reference solution, allowing the system to infer correct behavior and edge cases without human intervention.