ExecTune: Effective Steering of Black-Box LLMs with Guide Models

A research team from Amazon, UCLA, and other institutions has introduced ExecTune, a novel training framework designed to make AI agent systems more efficient and reliable. The work addresses a critical problem in deploying large language models (LLMs) via APIs like OpenAI's or Anthropic's: recurring inference costs can quickly surpass one-time training expenses. The proposed solution, called Guide-Core Policies (GCoP), uses a smaller, tunable 'guide' model to generate a structured plan or strategy. This plan is then executed by a larger, more capable, but expensive and opaque 'core' model (e.g., GPT-4, Claude Opus). The core innovation of ExecTune is its training recipe, which directly optimizes for 'executability'—the probability the core model can correctly follow the guide's plan—using a combination of teacher-guided sampling, fine-tuning, and reinforcement learning.

The results are significant for cost-sensitive AI applications. Across mathematical reasoning and code-generation benchmarks, systems using ExecTune achieved accuracy improvements of up to 9.2% over previous state-of-the-art methods while simultaneously reducing inference costs by up to 22.4%. This efficiency gain translated into dramatic model substitutions: Claude Haiku 3.5, a smaller, faster model, was able to outperform the larger and more expensive Claude Sonnet 3.5 on both math and coding tasks. Furthermore, Haiku guided by ExecTune came within 1.7% absolute accuracy of the top-tier Claude Sonnet 4 model, but at a 38% lower cost. Beyond pure efficiency, the GCoP framework supports modular updates, allowing the guide model to be improved or adapted without the need to retrain the massive, proprietary core model, offering a flexible path for continuous system enhancement.