Neuroevolution Arena shows RL wins training but loses ecological edge

A new preprint from Yuxu Ge and Yifei Cheng introduces Neuroevolution Arena, a GPU-accelerated spatial ecology where independently parameterized neural-network cells evolve and compete. The study crosses three update-and-inheritance regimes—EvoEvo (pure evolution), EvoRL (evolution plus reinforcement learning), and RLRL (pure RL)—with two neural architectures. Each condition runs for 50,000 generations across three independent training runs, producing 18 elite-controller artifacts. These artifacts are then evaluated in a frozen-evaluation design spanning 198 computational jobs, with three seed-defined ecological contexts: two cooperation-permitting and one attack-permitting.

The results are striking: RL-enabled regimes (EvoRL and RLRL) achieve significantly higher recorded training fitness than EvoEvo. However, pairwise ecological outcomes show architecture-conditioned majority patterns and substantial artifact dependence—meaning a model that dominates in training can lose in competitive ecological settings. Six-way winners vary across artifacts and contexts, and the prespecified survival endpoint hits a complete floor. The authors' key contribution is a nested protocol that separates training-run artifacts from evaluation contexts, exposing rather than concealing different sources of variation. This challenges conventional benchmarking, suggesting that training fitness alone is a poor predictor of real-world robustness.