An AI Teaching Assistant for Motion Picture Engineering
A 7-week study found AI tutors didn't affect exam scores, even when allowed during open-book tests.
Researchers from Trinity College Dublin, Deirdre O'Regan and Anil C. Kokaram, have published a detailed case study on implementing an AI Teaching Assistant (AI-TA) for a Master's program in Motion Picture Engineering. The system was built using a Retrieval-Augmented Generation (RAG) pipeline, a technique that grounds an LLM's responses in a curated knowledge base, to ensure accurate and course-specific answers. Over a 7-week pilot involving 43 students, the AI-TA handled 1,889 queries across 296 sessions, providing a substantial dataset for analysis.
A key and controversial finding was the experiment's inclusion of the AI-TA in open-book examinations. Statistical analysis across three exams showed no significant performance difference between students who had access to the tool and those who did not (p > 0.05). This suggests that thoughtfully designed assessments, likely focusing on higher-order thinking rather than rote recall, can maintain academic validity even when students use AI assistants. Student feedback was positive on utility (mean rating 4.22/5) but ambivalent on preferring it over human tutoring (mean 2.78/5), indicating AI's role as a supplement, not a replacement.
- The AI-TA used a RAG pipeline to provide accurate, course-specific answers to technical questions.
- A 7-week study with 43 students generated 1,889 queries, showing high adoption and utility.
- Crucially, allowing AI-TA use in exams did not alter student performance, challenging fears about AI-assisted cheating.
Why It Matters
Provides a real-world blueprint for integrating AI tutors in technical education while preserving academic integrity in assessments.