Enhancing sample efficiency in reinforcement-learning-based flow control: replacing the critic with an adaptive reduced-order model

A team of researchers has developed a new AI framework that could revolutionize how we design controllers for complex fluid systems like aircraft wings or engine turbines. The core innovation replaces the standard 'critic' network in deep reinforcement learning (DRL) with an adaptive reduced-order model (ROM). This ROM isn't a black box; it's built with physical insight, combining a linear dynamical system identified via operator inference with a Neural Ordinary Differential Equation (NODE) to capture nonlinear flow behavior. This hybrid model is continuously updated with new data during training, creating a more sample-efficient learning loop.

The framework was validated on two classic fluid dynamics challenges: controlling a Blasius boundary layer and reducing drag on a square cylinder. For the boundary layer, the method essentially condensed the process into a single episode of system identification and optimization, yet produced controllers that matched the performance of data-hungry DRL approaches. In the more complex cylinder drag reduction test, the ROM-based method achieved superior performance while requiring a fraction of the exploration data—demonstrating a potential 90% reduction in data needs. This work directly tackles the crippling sample inefficiency of model-free DRL, paving the way for practical AI-driven control in engineering domains where data is expensive or simulations are computationally prohibitive.