Multi-Agent LLM Architecture Triples Financial Literacy Assessment Accuracy

A team of researchers from an undisclosed institution has developed Agentic BKT, a multi-agent large language model architecture designed to assess financial literacy in real time without interrupting gameplay. The pipeline processes open-ended events from a 2D platformer serious game aligned with the OECD/INFE financial literacy framework. It operates in four phases: capturing player decisions as structured event logs, classifying actions via an LLM on a four-point rubric (validated with substantial inter-rater agreement, Fleiss kappa = 0.624), then passing behavioral trajectories to four domain-specialized agents covering risk mitigation, investing, spending, and credit management. Each agent performs session-level reasoning and feeds into a Bayesian Knowledge Tracing model. Finally, an expert judge agent synthesizes domain-level mastery into a single score.

Evaluated on 193 K-12 participants across 264 game sessions, the Agentic BKT pipeline demonstrated convergent and discriminant validity: mastery estimates correlated significantly with learning gain (r = 0.276, p = 0.0001) and post-test scores (r = 0.333, p < 0.0001), but showed no correlation with pre-test scores. Most notably, the multi-agent approach tripled the predictive validity of a single-LLM baseline (r = 0.095, not significant), proving that domain decomposition and session-level reasoning are critical for capturing the multidimensional nature of financial literacy. This research, submitted to IEEE CoG 2026, opens the door for stealth assessment in educational games without disrupting the learner's experience.