AdaEvolve: Adaptive LLM Driven Zeroth-Order Optimization

A research team from UC Berkeley, Stanford, and UT Austin has introduced AdaEvolve, a breakthrough framework that transforms how Large Language Models (LLMs) are used for automated program generation and optimization. The system addresses a critical limitation in current LLM-driven evolutionary approaches: static resource allocation that wastes compute on stagnating populations while under-exploring promising solutions.

AdaEvolve operates through three adaptive layers. Local Adaptation dynamically modulates exploration intensity within individual solution populations using accumulated improvement signals. Global Adaptation employs bandit-based scheduling to route computational budgets across different solution populations, ensuring resources flow to the most promising areas. Meta-Guidance generates novel solution tactics when progress stalls, creating new strategies based on previously generated solutions and their improvements.

The framework represents a shift from one-shot LLM generation to intelligent, inference-time search where LLMs function as semantic mutation operators within evolutionary loops. In testing across 185 diverse optimization problems—including combinatorial challenges, systems optimization, and algorithm design—AdaEvolve consistently outperformed open-source baselines while reducing computational waste by approximately 40%. This efficiency gain comes from the system's ability to recognize non-stationary search dynamics and reallocate resources accordingly.

For practical applications, AdaEvolve enables more efficient automated discovery of algorithms, system configurations, and mathematical solutions. The hierarchical approach allows researchers to tackle complex optimization problems that previously required prohibitive computational resources, opening new possibilities for AI-driven scientific discovery and engineering design.