New AI debugging method boosts Dafny fault localization by 3x

A new arXiv paper explores automated fault localization for verification-aware languages like Dafny, where verification failures often point to symptoms rather than root causes. The researchers compared state-based localization inspired by AutoFix against counterexample-based strategies that progressively leverage verifier output. Using a MutDafny-generated mutant dataset from DafnyBench and the EXAM score for evaluation, they found counterexample-based approaches clearly outperformed state-based ones. Structured ranking over a single trace gave the biggest boost over raw counterexample extraction, while aggregating multiple traces added further robustness by reducing solver path bias and expanding coverage. The takeaway: effective fault localization in verification-aware languages depends not just on using counterexamples, but on how that information is structured and diversified.