Breaking Validity-Induced Boundaries to Expand Algorithm Search Space: A Two-Stage AST-Based Operator for LLM-Driven Automated Heuristic Evolution

Researchers Sun Shengming and Shi Jialong have developed a breakthrough method that fundamentally changes how Large Language Models (LLMs) evolve algorithms. Their paper introduces a two-stage evolutionary operator that intentionally breaks traditional constraints requiring generated code to be immediately valid. In the first stage, the system performs crossover and mutation operations directly on Abstract Syntax Trees (ASTs) of heuristic code, deliberately creating diverse but often invalid structural variants. This radical approach allows exploration of algorithm spaces that conventional methods cannot reach.

In the second stage, LLMs like GPT-4 or Claude are employed to repair these invalid heuristics into executable, high-quality code. The repaired algorithms are then integrated into the evolutionary population, preserving valuable structural patterns that would have been discarded in traditional approaches. Experimental results demonstrate significant improvements: the method enhanced optimization performance and convergence speed on benchmark problems including the Traveling Salesman Problem (TSP) and Online Bin Packing Problem (OBP), outperforming state-of-the-art LLM-AHD frameworks like EoH-S.

The research addresses a fundamental limitation in current LLM-driven automated heuristic design, where end-to-end generation restricts exploration to the model's pre-trained knowledge. By separating structure exploration from code validation, the method enables more creative algorithm discovery while maintaining practical utility. This approach could accelerate development of optimization algorithms for complex real-world problems in logistics, scheduling, and resource allocation.