Preemptive Solving of Future Problems: Multitask Preplay in Humans and Machines

Humans excel at preemptively solving future problems by applying lessons from one task to another they haven't yet tackled. This counterfactual reasoning capability has long been difficult to replicate in AI. Now, a team of researchers from University of Michigan and MIT—Wilka Carvalho, Sam Hall-McMaster, Honglak Lee, and Samuel Gershman—has formalized this process in a new algorithm called Multitask Preplay. The algorithm replays experience from a completed task as a starting point for 'preplay,' or counterfactual simulation, of an accessible but unpursued task. This builds a predictive representation that enables rapid, adaptive performance when that new task is eventually encountered.

In controlled experiments with human participants navigating grid-world environments, Multitask Preplay better predicted how people generalized to tasks they had access to but never performed—even when participants were unaware they would need to generalize. The model then scaled successfully to Craftax, a complex, partially observable 2D Minecraft-like environment. When embedded into artificial agents, Multitask Preplay enabled them to learn behaviors that transferred to novel Craftax worlds sharing task co-occurrence structures, significantly outperforming baseline agents. The findings suggest that endowing AI with this human-like preemptive learning mechanism can dramatically improve performance in challenging multitask settings, bringing artificial intelligence closer to the flexible generalization humans exhibit naturally.