From Heuristic Selection to Automated Algorithm Design: LLMs Benefit from Strong Priors

A team of researchers from Leiden University and other institutions has published a significant paper titled "From Heuristic Selection to Automated Algorithm Design: LLMs Benefit from Strong Priors" on arXiv. The work addresses a key limitation in current LLM-driven automated algorithm design, where models typically rely on adaptive prompt engineering without structured guidance from proven solutions. The researchers demonstrate that by providing LLMs with high-quality algorithmic code examples—termed "strong priors"—the models can generate significantly better optimization algorithms. This approach moves beyond simple heuristic selection toward more systematic automated design.

The technical breakthrough involves analyzing token-wise attribution in prompts to understand how example code influences LLM outputs. The team tested their method on two established black-box optimization benchmarks: the pseudo-Boolean optimization suite (pbo) and the black-box optimization benchmark (bbob). Results showed superior performance compared to standard prompt-based approaches, with improvements in both efficiency and robustness. This research highlights the value of integrating benchmarking studies and existing algorithmic knowledge into LLM workflows, potentially accelerating development in fields like evolutionary computing, automated machine learning (AutoML), and complex system optimization where algorithm design is critical.