Evolutionary LLM method boosts PINN design by 95%

Physics-informed neural networks (PINNs) are notoriously sensitive to interacting choices of architecture, activation functions, loss weighting, and optimization hyperparameters. While large language models (LLMs) can propose configurations, they typically don't learn from previous training results. To overcome this, Xu Yang and colleagues developed a closed-loop evolutionary algorithm that uses measured training outcomes to guide an LLM's next-generation suggestions. The system maintains an evaluated population, applies parent-conditioned mutation and crossover, preserves elite solutions, and converts successes and failures into context for the LLM. Every proposed configuration is executed under a fixed optimizer-step budget, enabling direct performance comparison.

On a challenging 1D multiscale wave equation, two independent 10-generation runs trained 60 PINNs for 600,000 optimizer steps total. In both runs, the best configuration emerged in the final generation: one run achieved a 2.97% MSE reduction, while the stronger run reached a dramatic 95.38% reduction. The top design validated residual connections, increased depth on separate branches, combined them in later generations, and refined width and collocation density. Notably, the algorithm revealed that low solution error can coexist with a high PDE residual—a counterintuitive insight. These results show that evolutionary-guided LLMs can effectively automate PINN architecture search, paving the way for broader physics-aware evaluation.