New study designs 9 AI demos to track student LLM use in engineering

A new arXiv preprint (2607.28710) tackles a pressing question in higher education: how are engineering students actually using LLMs like ChatGPT, and what does that mean for learning? The 47-page paper, authored by Shuang Geng, Emma Lejeune, and colleagues, presents a descriptive study design from an undergraduate engineering mechanics course taught in Spring 2026. Rather than relying on anecdotal reports, the team built a reproducible survey instrument to capture student AI usage patterns, attitudes, and verification practices—then linked those responses to academic performance metrics.

The study also documents a deployable sequence of nine structured, instructor-led AI demonstrations designed to teach students strategic delegation and evaluation of LLM outputs. These demos model how to assign tasks to AI, check responses, and decide when not to use AI at all. Preliminary data from the course reveal shifting student behaviors and a complex relationship between AI reliance and outcomes—one that defies simple narratives of AI helping or hurting grades. The researchers emphasize that their main contribution is the open-access methodological framework: they provide the complete study design, survey tools, and demonstration materials so other engineering educators can collect and share comparable data.

This work highlights an urgent need for domain-specific empirical evidence to guide classroom AI policies. With widespread student adoption outpacing institutional guidance, the authors argue that a collaborative, evidence-based approach is essential to understanding AI's long-term impact on learning. The paper includes 10 figures and 9 tables, offering a detailed template for instructors who want to move beyond guesswork in shaping AI pedagogy.