New paper reveals AI agents can pick wrong research candidates despite higher aggregate scores

A new paper from researchers Adithya Srinivasan and Devesh Paragiri exposes a critical flaw in how autonomous research agents evaluate candidates. These agents typically optimize a single aggregate metric (e.g., global average) across heterogeneous spaces. The authors show that when scientific validity is defined by disaggregated structure—like per-region or per-cohort performance—the aggregate can rank the wrong candidate first. They call this the 'inversion' problem: the headline number improves while the underlying structure inverts, so a decision based on the number accepts a candidate that breaks the model. They demonstrate this on an Ecosystem Demography model for fire prediction: the top-scoring candidate was within noise of a slightly lower-scoring one on global score, yet it collapsed protected boreal regions while the lower-scoring candidate preserved them. The failure is domain-agnostic, appearing wherever candidates have multi-dimensional validity but a single reduction is used as verifier.

To solve this, the authors propose a 'search-discipline protocol' that moves the final decision to an external control loop. This loop audits each candidate on its disaggregated behavior after the agent has produced and ranked candidates. It can demote a candidate the agent would have accepted, and it can reopen a run the agent had declared finished. The key insight: the agent optimizing the score is the last party likely to catch the score being wrong, and a prompt has no remaining turn once the agent stops. The protocol decides on reviewable candidate-effect evidence instead of the score. This approach offers a practical safeguard for long-horizon research agents operating in high-stakes scientific domains where regional or subgroup impacts matter more than a single headline number.