Theory of Mind in Action: The Instruction Inference Task in Dynamic Human-Agent Collaboration

A team of researchers from North Carolina State University and the University of Illinois Urbana-Champaign has published a groundbreaking study on AI's ability to understand human intentions, a cognitive skill known as Theory of Mind (ToM). They introduced a novel benchmark called the Instruction Inference Task, designed to test how well AI agents can interpret incomplete or ambiguous instructions in dynamic, goal-oriented collaborations. To tackle this, they created Tomcat, an LLM-based agent framework, and implemented two variants: Fs-CoT (few-shot chain-of-thought) and CP (commonsense prompt).

They rigorously tested Tomcat on three leading large language models: OpenAI's GPT-4o, DeepSeek's DeepSeek-R1, and Google's Gemma-3-27B. The study involved 52 human participants who performed the same task, providing a direct performance benchmark. The team measured success using three metrics: intent accuracy (correctly inferring the goal), action optimality (choosing the right steps), and planning optimality (efficiently reaching the goal).

The results were significant. The Tomcat agent using the Fs-CoT prompting method, particularly when powered by GPT-4o and DeepSeek-R1, achieved performance levels comparable to the human participants. This demonstrates that modern LLMs, when properly prompted, can exhibit a form of practical ToM reasoning essential for effective human-AI teamwork. The study, spanning 66 pages with extensive appendices, provides a new framework for evaluating and developing AI agents that can truly collaborate by understanding context and unspoken intentions.