Study: Hands-on GenAI learning beats lectures for metacognitive skills

A new short-term longitudinal study from researchers at a European university (Pau Benazet i Montobbio, Janne Rotter, Davinia Hernández-Leo) investigates how teaching approach affects metacognitive awareness in AI-assisted learning. The quasi-experiment divided 126 first-year engineering students into two groups: one experienced a two-hour hands-on experiential learning session with generative AI, the other received a traditional instructional lecture on the same topic. Metacognitive awareness—split into knowledge of cognition (knowing effective AI strategies) and regulation of cognition (applying those strategies)—was measured before, immediately after, and five weeks post-intervention.

The results reveal a nuanced timeline. Immediately after the session, the experiential group showed significantly higher engagement and knowledge of cognition compared to the instructional group. However, by the five-week follow-up, both groups had converged on these measures. The key differentiator emerged in regulation of cognition: only the experiential group demonstrated a delayed, continuous within-group increase in this skill over the five weeks, suggesting that the benefits of hands-on learning take time to crystallize into practical self-regulation.

These findings have immediate implications for AI education. The study demonstrates that experiential learning—where students actively prompt, critique, and iterate with GenAI tools—fosters deeper metacognitive development than passive instruction, especially in the critical skill of regulating one's own AI use. The delayed effect on regulation of cognition hints that knowledge and regulation develop on different timescales, with regulation requiring an incubation period that only experiential methods seem to trigger.

For companies and educators designing AI upskilling programs, this research supports moving beyond lectures to structured, hands-on practice sessions. The study also opens questions about how to sustain and accelerate the regulation gains observed in the experiential group, potentially through spaced repetition or guided reflection exercises. As GenAI becomes ubiquitous in workplaces, building users' metacognitive awareness is essential for productive, safe, and ethical AI adoption.