RL4F benchmark brings offline RL to nuclear fusion plasma control

Researchers have released RL4F (Reinforcement Learning for Fusion), a standardized offline reinforcement learning benchmark for plasma control in nuclear fusion. The benchmark was built using historical discharge data from DIII-D, a real-world tokamak operated by General Atomics. It covers four full-profile tracking tasks—rotation, density, temperature, and pressure—each requiring long-horizon, multi-actuator control. The team evaluated a broad set of imitation learning and offline RL baselines under a unified protocol. They found that offline model-based RL methods obtain the best average performance on most objectives, though no single method dominates all tasks. This highlights the importance of dynamics modeling in complex plasma control problems.

The RL4F benchmark addresses a critical gap: without a standardized evaluation framework, progress in offline RL for fusion has been difficult to measure. The authors open-sourced the codebase, datasets, and evaluation environment to foster further research. The paper, authored by Yang Fu, Haomin Bao, and colleagues, spans 23 pages and is available on arXiv. By providing closed-loop evaluation environments and baseline comparisons, RL4F aims to accelerate the development of safer, data-driven plasma controllers—potentially reducing the need for costly and risky online trial-and-error on real fusion devices.