Thermodynamics of Reinforcement Learning Curricula

A team of researchers has published a novel paper, 'Thermodynamics of Reinforcement Learning Curricula,' that applies principles from non-equilibrium thermodynamics to formalize curriculum learning in reinforcement learning (RL). The work, by Jacob Adamczyk, Juan Sebastian Rojas, and Rahul V. Kulkarni, introduces a geometric framework where the parameters defining an RL task's reward function are treated as coordinates on a high-dimensional 'task manifold.' The core theoretical insight is that the most efficient sequence for training an AI agent on progressively harder tasks—the optimal curriculum—corresponds to the shortest path, or geodesic, across this manifold. This path is found by minimizing the 'excess work,' a concept borrowed from thermodynamics that quantifies the inefficiency of moving a system (here, the learning agent) between states.

As a direct application, the researchers derived the 'MEW' (Minimum Excess Work) algorithm. MEW provides a mathematically grounded schedule for 'temperature annealing' in maximum-entropy RL, a popular technique that encourages exploration by initially allowing random actions and gradually reducing this randomness. Instead of relying on heuristic or manually tuned schedules, MEW calculates the annealing path that minimizes thermodynamic work, promising a more systematic and potentially faster route to a high-performing, specialized agent. The paper, which has been accepted at the SciForDL Workshop at the International Conference on Learning Representations (ICLR) 2026, represents a significant cross-disciplinary bridge, using physics to solve a core machine learning engineering challenge: how to train AI systems most efficiently.