DePro: Understanding the Role of LLMs in Debugging Competitive Programming Code
New research shows a test-case driven AI approach reduces debugging attempts by 64% and saves 7.6 minutes per problem.
A research team from academia has introduced DePro, a novel AI-powered system designed to tackle one of software development's most time-consuming tasks: debugging. Presented in a paper accepted to FSE 2026, DePro takes a unique test-case driven approach to assist programmers by correcting existing faulty code rather than generating new solutions from scratch. The system works by combining brute-force reference generation to create correct outputs, stress testing to identify discrepancies, and iterative refinement guided by a Large Language Model (LLM) to pinpoint and resolve errors.
In their empirical study, the researchers evaluated DePro on 13 faulty user submissions from the competitive programming platform Codeforces. The results were significant: DePro consistently produced correct final solutions, demonstrating a substantial improvement over both human programmers and standard zero-shot LLM debugging. The system reduced the number of debugging attempts required by up to 64% and slashed the average debugging time by 7.6 minutes per problem. This research provides a concrete benchmark for evaluating LLMs in debugging within the demanding context of competitive programming, which features diverse problem domains and strict efficiency requirements.
- DePro reduces debugging attempts by up to 64% on competitive programming code compared to humans and standard LLMs.
- The system cuts average debugging time by 7.6 minutes per problem using a test-case driven, iterative refinement process.
- It was validated on 13 faulty Codeforces submissions and consistently produced correct solutions, demonstrating reliable performance.
Why It Matters
This research provides a concrete, efficient framework for using AI to automate and accelerate a critical, time-intensive phase of the software development lifecycle.