ChatGPT in exams shifts assessment from answers to reasoning, study finds

A recent arXiv study (2605.12363, May 2026) by Qusay H. Mahmoud reimagines academic assessment in the age of generative AI. Rather than banning AI, the researcher allowed engineering students to use ChatGPT during take-home open-book exams—crucially requiring them to submit full interaction transcripts alongside their solutions. This methodology provided direct observational evidence of reasoning processes, bypassing self-reported data. The qualitative analysis identified three progressive patterns of AI use: basic answer retrieval (copying questions verbatim), guided collaboration (iterative prompt refinement), and critical verification (testing and evaluating AI outputs). Notably, the strongest evidence of student reasoning occurred when they encountered incorrect or incomplete AI responses—students demonstrated evaluative reasoning through debugging, comparison, and justification.

The findings fundamentally challenge traditional assessment assumptions. When generative AI is integrated transparently, the cognitive task shifts from producing solutions to assessing solution validity. Correct final answers alone no longer suffice as evidence of comprehension. Instead, competencies such as prompt formulation, verification, and judgment become visible indicators of learning. Students shifted focus from rule avoidance to self-regulation. Mahmoud argues that generative AI does not invalidate assessment but can expose deeper forms of understanding aligned with professional practice. The study recommends that assessments evolve to evaluate reasoning about solutions rather than independent solution production, suggesting a framework where AI is a collaborative tool that reveals cognitive processes rather than a cheating threat.