ProofAgent Harness exposes AI agent failures through adversarial multi-turn trials
A small, quantized local LLM just outperformed production-grade models in adversarial agent testing—not because it was smarter, but because the evaluation harness itself was designed differently.
Fouad Bousetouane’s new paper, ProofAgent Harness, presents open-source infrastructure designed to stress-test AI agents in high-risk environments. Unlike traditional evaluations that judge isolated outputs or static tasks, ProofAgent Harness runs adversarial multi-turn trials where agents must handle tools, retain context, follow policies, and interact over multiple turns. It employs Adversarial Multi-Juror Scoring with Turn-Level Audit, using calibrated juror personas, consensus checks, and turn-level evidence to produce evidence-linked reports. Researchers can extend domains, traps, metrics, juror personas, and reporting formats.
The harness was evaluated across four domains: customer support, medical triage, privacy and security, and code generation. Results showed that even strong agents fail selectively through weak metrics, fragile turns, unsafe reframing, and manipulation paths. Remarkably, a small quantized local Harness LLM could challenge production agents powered by large state-of-the-art LLMs, suggesting that evaluation capability arises from the full harness pipeline rather than model scale alone. ProofAgent Harness turns AI agent evaluation from a static score into scalable, repeatable, evidence-backed, and actionable adversarial infrastructure.
- ProofAgent Harness demonstrates that a small quantized local LLM can outperform large proprietary models in adversarial multi-turn evaluations, proving pipeline design matters more than model scale.
- Existing tools like LangSmith, PyRIT, and Garak focus on model-level or single-turn evaluation, leaving a critical gap for multi-turn agent assessment that ProofAgent Harness fills.
- Enterprises deploying autonomous agents must adopt adversarial multi-turn evaluation to uncover compounding errors; the hidden cost of not doing so could be catastrophic failures in production.
- The open-source nature of ProofAgent Harness lowers barriers to rigorous agent testing, but juror bias and computational expense are unresolved risks that need careful management.
Why It Matters
Shifts focus from model scale to evaluation pipeline design, reshaping how we assess agent reliability.