CoHarden beats SWE-bench with iterative test-fix hardening for bug repair

A new paper from Microsoft and Chinese universities tackles a hidden flaw in automated program repair with LLMs: bug reproduction tests (BRTs) that pass fail-to-pass checks but still allow incorrect patches. The team shows that some BRTs are 'lax'—they reproduce symptoms but are too weak to reject plausible-but-wrong fixes. This undermines the entire repair pipeline when tests and patches are co-generated, because the in-trajectory fail-to-pass check can pass even when both test and patch are wrong.

To fix this, the authors introduce CoHarden, an iterative co-generation framework that first generates a test from the bug report, then repeatedly hardens both the test and proposed fix against surviving mutation patches. The loop converges when the test no longer admits 'lax' regressions. CoHarden reaches 69.4% Resolved and 78.9% F→P on SWE-bench Verified, beating the strongest baselines by +9.6 and +7.9 percentage points (Resolved) respectively. The method generalizes across different LLM backbones and benchmarks, showing that rigorous BRTs are critical for real-world bug repair.