Fitting Reinforcement Learning Model to Behavioral Data under Bandits

A team of researchers has published a new paper, 'Fitting Reinforcement Learning Model to Behavioral Data under Bandits,' introducing a faster, more accessible method for analyzing decision-making. The work, led by Hao Zhu, Jasper Hoffmann, Baohe Zhang, and Joschka Boedecker, tackles a core problem in neuroscience, psychology, and economics: accurately fitting computational models of learning (like RL models) to the choices humans or animals make in uncertain, reward-based tasks (bandit problems). Their key innovation is a generic mathematical formulation of the fitting problem that enables the use of convex optimization techniques.

The researchers provide a detailed theoretical analysis proving the convexity properties of their formulation. This allows them to apply highly efficient convex optimization solvers, which are fundamentally faster and more reliable than the heuristic or approximate methods often used for this complex, non-convex problem. In evaluations across simulated and real-world bandit environments, their method matched the accuracy of state-of-the-art benchmarks but did so with a substantial reduction in computation time. To democratize access, the team has released their work as an open-source Python package, enabling any researcher to fit sophisticated RL models to their behavioral data without needing deep expertise in optimization theory.