AI Coding Agents: Contracts Boost Reviewability, Not Correctness

A new study from Vincent Schmalbach explores whether formal “software delegation contracts” can improve the output of AI coding agents. The paper, posted on arXiv, builds on prior theoretical work and runs a controlled pilot with 64 agent executions across two model tiers. Agents were given 10 different API tasks with seeded defects and missing docs under three conditions: a realistic issue prompt, an explicit contract, and a contract requiring an evidence bundle. Each run was scored via hidden tests, mutation checks, scope analysis, and three blinded reviews. The results are striking: every single run passed all acceptance tests with zero scope violations, meaning the agents already delivered correct code under simple prompts. However, explicit contracts dramatically improved the *reviewability* of that code. Evidence sufficiency jumped +0.83 on a 5-point scale (p < 0.0001, Cliff’s delta = 0.66) across 22 of 30 paired comparisons, while reviewer ambiguity fell (p = 0.035). Components like changed-file lists, known-limitations sections, and residual-risk checklists appeared only when demanded by the contract.

This improvement came at a cost: contracts consumed 13% more tokens and 38% more wall-clock time, with weaker models hit harder. The study suggests that for small, well-defined tasks, delegation contracts buy transparency and auditability—not correctness—since the agents already get the code right. For teams integrating AI coding agents into production workflows, this means contracts are a valuable tool for human oversight, allowing reviewers to quickly understand what changed and what risks remain. The authors note the study is a small pilot and call for larger experiments on complex, real-world repositories to see if contracts also reduce bugs or improve maintainability. For now, the takeaway is clear: if you want your AI coder to be accountable, give it a contract.