Anthropic's Opus 4.6 agents double performance with simple elicitation prompts

A new LessWrong post from researchers zef, kaivu, leni, and rohuang introduces a case study on inverse rubric optimization (IRO), where LLM agents must learn the preferences of a black-box judge under a label budget. They decompose agent performance into two key attributes: efficiency (how well each resource is used) and propensity (how much resource the agent consumes before stopping). Default agents, they find, are under-elicited—meaning they don't fully exploit their resource budget or optimize efficiently.

The researchers test general prompt interventions (learnability cues, iteration strategies, and their combination) and scaffold changes using Anthropic's Opus 4.6 as the optimizer and Haiku 4.5 as the poem generator. Results show that simple, task-agnostic prompts roughly double the final evaluation score at every resource budget (e.g., 10k labels). Elicited runs climb much steeper per label and continue optimizing far longer than baselines. The work suggests that significant AI performance gains are achievable through prompt elicitation without model retraining.