New RL Technique Lets You Train the LLM Agent's Harness, Not the Model

Large language model agents are typically improved by tweaking prompts, swapping models, or hand-crafting workflows. The underlying execution harness – the infrastructure that orchestrates tool calls, loops, and error handling – is treated as fixed. A new paper from researchers Haiwen Yi and Xinyuan Song challenges this assumption, arguing that the harness itself is a learnable control layer. They formalize harness operation as a finite-horizon Harness MDP, where a lightweight controller selects structural execution actions while the LLM executor remains frozen. The controller is trained from offline rollouts using advantage-weighted regression with only terminal task-rubric rewards. Crucially, they separate final task quality from a post-hoc Harness Maturity Score, which measures whether the harness follows reliable execution patterns rather than only whether the final answer is correct. This separation gives a finite-buffer view of harness learning: final-quality gains require high-return support in the offline buffer, while process behavior can shift whenever it aligns with advantage-weighted actions.

Across six controlled domains and two public-benchmark adapters, the learned controller consistently improves verification behavior and selectively improves final task quality, with the largest gains on adapted tau-bench retail, adapted AgentBench DB-Bench, and coding with a calibrated structural verifier. Ablations against behavior cloning and Forced CHECK show that the gains are not explained by imitation or by simply adding checks. The results identify harness control as a learnable layer for frozen LLM agents, while showing that offline support limits when better process control becomes better final answers. This approach could enable more reliable and adaptable LLM agents without costly model retraining, simply by optimizing the orchestration layer.