New arXiv paper: Verifying AI code is now harder than writing it

A new paper from Binghai Wang and 11 other researchers, published on arXiv (2606.26300), argues that the classical intuition 'verifying is easier than producing' no longer holds for today's coding agents. As foundation models improve reasoning and engineering harnesses become more sophisticated, generating complex candidate solutions has become trivial — but reliably verifying them has become the harder problem. The authors introduce the concept of the 'verification horizon', noting that every verifier is only a proxy for human intent, never the intent itself. This creates a twofold difficulty: intent is inherently underspecified, and optimization during training widens the gap between proxy and intent, leading to reward hacking or signal saturation.

The researchers characterize verification signal quality along three dimensions — scalability, faithfulness, and robustness — and argue that achieving all three simultaneously is the central challenge. They study four reward constructions: a test verifier for general coding tasks, a rubric verifier for frontend tasks, the user as verifier for real-world agent tasks, and an automated agent verifier for long-horizon tasks. Their experiments show that targeted verification design can suppress reward hacking and improve task completion, but no fixed reward function remains effective as policy capability grows. The core takeaway: verification must co-evolve with the generator, making adaptive reward systems essential for building reliable coding agents.