LLM-Meta-SR: In-Context Learning for Evolving Selection Operators in Symbolic Regression

A research team from Victoria University of Wellington and Michigan State University has developed LLM-Meta-SR, a novel meta-learning framework that enables large language models (LLMs) to automatically design and evolve selection operators for symbolic regression algorithms. Symbolic regression involves discovering mathematical expressions that best fit given data, a task traditionally requiring significant human expertise. The researchers identified two key limitations in existing LLM-based algorithm evolution: lack of semantic guidance (leading to ineffective code component exchange) and code bloat (resulting in unnecessarily complex components that reduce interpretability).

To overcome these challenges, the team enhanced their framework with two key innovations: a complementary, semantics-aware selection operator and bloat control mechanisms. They also embedded domain knowledge directly into the prompts, enabling the LLM to generate more effective and contextually relevant selection operators. Experimental results on symbolic regression benchmarks show that LLMs can devise selection operators that outperform nine expert-designed baselines, achieving state-of-the-art performance.

Remarkably, the evolved operator further improved a state-of-the-art symbolic regression algorithm, achieving the best performance among 28 symbolic regression and other machine learning algorithms across 116 regression datasets. This demonstrates that LLMs can exceed expert-level algorithm design for symbolic regression, potentially accelerating scientific discovery by automating the creation of interpretable mathematical models from complex data. The approach represents a significant step toward using AI to design better AI systems, particularly in domains requiring both performance and interpretability.