LLMs evolve optimization algorithms 60x faster than manual design
900 algorithms generated, beating state-of-the-art at a fraction of the cost.
A team of researchers from Leiden University and TU Delft have demonstrated that large language models can generate better optimization algorithms than human experts — and do it 60 times faster. By extending the LLaMEA framework to multi-objective Bayesian optimization (MOBO), they used LLMs as genetic operators (mutation and crossover) within an evolutionary strategy, automatically producing complete algorithm implementations. The system integrated SMAC hyperparameter tuning into the evolutionary loop, resulting in roughly 900 candidate algorithms across nine evolutionary runs.
When tested on 12 synthetic problems (ZDT, DTLZ, WFG families) and three real-world engineering problems (RE), the best evolved algorithm consistently outperformed the state-of-the-art baseline qParEGO. On synthetic benchmarks, it achieved a mean normalized hypervolume of 0.971 (vs 0.869) while requiring roughly 60x less wall-clock time. On unseen real-world problems, it hit 0.985 (vs 0.971) at 3.4x lower cost, with statistically significant wins on 2 of 3 problems. This work shows that LLM-driven evolutionary search can discover Pareto-optimal algorithm designs that would be impractical to reach through manual design, potentially automating a key aspect of AI research.
- Evolved algorithm reached 0.971 hypervolume on synthetic benchmarks vs 0.869 for human-designed qParEGO, running 60x faster.
- On real-world engineering problems, the LLM-generated algorithm scored 0.985 vs 0.971 for the baseline, at 3.4x lower cost.
- System generated ~900 algorithm variants across 9 evolutionary runs, using LLMs as mutation/crossover operators within the LLaMEA framework.
Why It Matters
Automates algorithm discovery — LLMs can design better optimization methods than humans, at massive speed and cost savings.