Hierarchical RL-MPC Control for Dynamic Wake Steering in Wind Farms

A new research paper from Marcus Binder Nilsen and colleagues presents a hierarchical framework that combines reinforcement learning (RL) with model predictive control (MPC) for optimizing wind farm wake steering. The challenge of wake steering—adjusting turbine orientations to minimize power loss from downstream wakes—is notoriously difficult due to complex flow physics and rapidly changing wind conditions. Traditional MPC requires accurate state estimates, which are often unavailable in real-world settings. The proposed solution uses an RL agent to learn compensatory state estimates for the MPC controller, rather than directly controlling the turbines. This hybrid approach was evaluated on a three-turbine case study, where it achieved a 23% power gain over the baseline control method. Notably, it even surpassed the performance of an idealized MPC that had perfect knowledge of the system state, demonstrating that the RL agent can effectively compensate for model inaccuracies.

The study also compared the hybrid architecture to a direct RL control approach. While direct RL can achieve high performance, it often suffers from unstable control actions and safety concerns during training. The hierarchical RL-MPC framework maintained superior safety characteristics throughout the training process, with more stable control actions and comparable final performance. This makes it more practical for real-world deployment, where safety and reliability are paramount. The paper, submitted to IFAC for publication, is available on arXiv as arXiv:2604.22797. The work highlights a promising direction for combining learning-based methods with traditional control theory to tackle complex, real-world optimization problems in renewable energy systems.