A-GRPO: New AI control method hits 75.4% feasibility in hybrid powertrains

Control systems with terminal constraints—like hitting a final state-of-charge target on a hybrid vehicle—present a tricky problem for reinforcement learning. When feasibility is only checked at the final step, sparse rewards destabilize critic-based advantage estimation and weaken standard Lagrangian penalty approaches. To solve this, Md Ragib Rownak, Sidra Ghayour Bhatti, and Qadeer Ahmed introduced A-GRPO (Advantage-Ranked Group Relative Policy Optimization), a critic-free policy gradient method that augments group-relative updates with trajectory-level ranking. A Transformer-encoder actor reweights advantages based on constraint satisfaction, while a scale-adaptive normalization bounds advantage variance at every timestep. The paper demonstrates three formal properties: per-timestep variance boundedness, strict separation of feasible vs. violating trajectories under a verifiable ranking-weight condition, and bounded adaptive dual variables with drift-balance feedback.

On a 3,605-step series-hybrid powertrain energy management task with a terminal state-of-charge constraint, A-GRPO achieved 75.4% mean sustained feasibility with return within 3.7% of the dynamic programming optimum. That dramatically outperforms a PPO-Lag baseline, which only achieved 27.4% sustained feasibility. Ablation experiments confirm both the ranking mechanism and Lagrangian dual updates are necessary for this performance. The paper, accepted for presentation at the 2026 IEEE Conference on Decision and Control (CDC), provides a principled route to enforcing hard constraints in real-world control problems without needing a learned critic. A-GRPO could be especially valuable for energy management, robotics, and autonomous systems where terminal constraints matter and simulation-to-real transfer is required.