Evolutionary AI Discovers Optimal Reward Schedules for Deep RL Training
Novelty emerges as the dominant early reward signal, boosting performance by 11.4%
A new paper accepted at the 2026 IEEE International Conference on Development and Learning (ICDL) presents an evolutionary framework for automatically discovering developmental reward schedules in deep reinforcement learning. Traditionally, RL reward composition is hand-designed and fixed throughout training. The authors introduce three biologically inspired motivational components—agency, novelty, and reactivity—whose weights shift over time via evolution. They evaluated four evolutionary algorithms (CMA-ES, xNES, DE, L-SHADE) against an extrinsically motivated baseline and three hand-designed methods on two sparse-reward MiniGrid environments: DoorKey-6x6 and KeyCorridorS3R1.
Results show that evolved schedules consistently outperform static baselines. On DoorKey-6x6, L-SHADE achieved the best performance with an approximate mean improvement of 11.4% over the extrinsic-only baseline. On the harder KeyCorridorS3R1 task, CMA-ES excelled while other evolved methods showed weaker generalization. A surprising finding: novelty emerged as the dominant signal early in training across both tasks, contrary to the expected developmental ordering. This suggests that evolution in computational settings may diverge from biological optimization, positioning evolutionary reward schedule discovery as a promising direction for accelerating RL in complex environments.
- L-SHADE algorithm achieved 11.4% relative improvement over hand-designed reward baselines on DoorKey-6x6
- CMA-ES performed best on the more challenging KeyCorridorS3R1 task, with other methods showing weaker generalization
- Novelty consistently dominated as the early training signal, contradicting the assumed biological developmental ordering
Why It Matters
Automated reward schedule design could dramatically accelerate RL training in robotics and game AI without human hand-tuning.