Deep RL co-design framework for aeroelastic systems boosts performance

A new paper by Yao Cheng Li and Urban Fasel presents a model-free nested co-design framework for aeroelastic systems using deep reinforcement learning (DRL). Traditional control co-design treats physical system and controller separately, but this approach exploits their strong coupling. The framework trains a design-conditioned control policy with proximal policy optimization (PPO) while an outer loop iteratively updates a distribution over candidate design parameters. This enables simultaneous optimization of structural, aerodynamic, and control choices without requiring an explicit system model.

The method is evaluated on three case studies of increasing complexity: a spring-mass-damper, a pitch-plunge-flap aerofoil, and a highly flexible high-aspect-ratio glider performing a thermal-soaring mission in a stochastic environment. Across all tests, the framework concentrates the design search toward high-performing regions and outperforms policies trained on randomly sampled designs. Key findings show that reward shaping is essential for stable learning in partially observed and stochastic settings. For the glider, the approach jointly addresses wing design, flight control, and mission-level behavior under aeroelastic coupling and atmospheric uncertainty. These results demonstrate the promise of model-free co-design for complex systems where design, control, and mission objectives are deeply intertwined.