New behavioral embedding predicts how fine-tuning LLMs on games transfers skills

A new paper from researchers at the University of Waterloo, led by Joshua Caiata and including Sreepriya Pulyassary, Xiang Li, and Kate Larson, tackles a core problem in training large language models (LLMs): fine-tuning on one strategic task can either improve or impair an agent's ability to reason in other tasks, and this transfer is hard to predict. The team used normal-form games—standard game theory settings with explicit payoffs and equilibrium behavior—to study how LLM strategic capabilities shift after fine-tuning on specific games.

The authors propose a lightweight two-feature embedding that captures the behavioral demands of a game: the entropy of the Nash equilibrium (how 'random' the equilibrium strategy is) and the sensitivity of optimal responses to an opponent's action. They show that existing published structural embeddings, which encode payoff geometry or game identity, fail to generalize when making predictions on new games. In contrast, their behavioral embedding reliably predicts performance changes on held-out games, suggesting that what transfers between games is not the payoff structure but the underlying decision-making behavior required. This insight offers a practical way to forecast which training games will help or hurt an LLM's future strategic reasoning, and points to more efficient, targeted fine-tuning for multi-agent AI systems.