A Rubric-Supervised Critic from Sparse Real-World Outcomes

A team from Carnegie Mellon University and UIUC has published a novel paper, 'A Rubric-Supervised Critic from Sparse Real-World Outcomes,' addressing a critical gap in AI agent development. Current benchmarks for coding agents, like SWE-bench, reward autonomous task completion with verifiable metrics such as unit-test success. However, real-world agents collaborate with humans, where success signals are often noisy, delayed, and sparse. The researchers propose bridging this gap by learning a 'critic' model—a model that can evaluate agent performance—from this imperfect, real-world interaction data. This critic can then be used as a reward model for reinforcement learning or for inference-time scaling methods like best-of-N reranking.

The core innovation is 'Critic Rubrics,' a semi-supervised framework defined by 24 behavioral features that can be automatically derived from interaction traces (e.g., code edits, command usage) without requiring constant human scoring. The model is trained to jointly predict these rubric scores and any available sparse human feedback. In experiments, critics trained this way significantly outperformed baselines. They improved best-of-N reranking on the SWE-bench coding benchmark by +15.9% (Best@8 vs. Random@8), enabled effective early stopping that achieved +17.7 performance with 83% fewer agent attempts, and supported better training data curation. This work provides a scalable pathway to align AI agents with nuanced, real-world human preferences beyond simplistic pass/fail tests.