New arXiv paper slashes LLM prompt search costs 5.6–54x with tiered evolution

Evolutionary optimization of LLM prompts—like the GEPA algorithm—relies on repeatedly evaluating candidate generations, and those fitness calls quickly become the dominant cost. A new arXiv paper (2608.10694) from Tal Oved, Roi Pony, Oshri Naparstek, and Udi Barzelay tackles this by decoupling the three roles an LLM plays in the search loop. High-volume answering and fitness scoring run on the cheapest model tier, while a strong, expensive model is reserved for the rare reflection and variation operations that generate new candidates. The cheaply evolved prompt is then transferred upward to a stronger target model for deployment, a strategy the authors call 'upward cross-tier transfer.'

Tested across HotpotQA, IFBench, LiveBench-Math, and HoVer, with 11 models spanning four families, the approach consistently matches or beats same-tier optimization while placing over 96% of total search tokens on the cheapest tier. The result is a 5.6–14x reduction in search cost, and a striking 25–54x reduction when reasoning-tier models emit long chains of thought on every fitness call. The paper also provides a cost-controlled analysis of when cheap-tier substitution works and where it breaks down. For practitioners, this makes evolutionary prompt search dramatically more accessible, letting teams iterate on budget and deploy on frontier models without sacrificing quality.