Researchers propose TrajSpec to improve AI bug fixing
New method TrajSpec boosts AI program repair accuracy by up to 73% using evidence-driven bug report refinement.
A team of researchers led by S M Farah Al Fahim from Concordia University has developed TrajSpec, a novel method to refine bug reports for automated program repair (APR) agents. The approach addresses a critical gap in bug reports, which often lack repair-relevant details such as failure-inducing behavior or behavioral requirements. TrajSpec uses a trajectory-collection agent to gather evidence from the repository, organizing it into a three-level hierarchical structure: high-level interpretation, diagnostic findings, and concrete observations. This refined report is then reviewed to remove unsupported claims and add repository-backed details.
In evaluations on all 300 instances of SWE-Bench Lite using Mini-SWE-Agent V2, TrajSpec improved Pass@1 (a metric for first-attempt repair success) from 41.00% to 59.67% with GPT-5-mini and from 54.67% to 64.33% with MiniMax M2.5. On a stratified sample of 100 instances, it boosted Pass@1 from 41.00% to 71.00% with Agentless and from 47.00% to 72.00% with AutoCodeRover. Ablation studies confirmed the importance of repository-based review and hierarchical evidence representation, which contributed to the observed performance gains.
- TrajSpec improves automated program repair (APR) success rates by up to 73% by refining bug reports with repository-supported evidence.
- Tested on SWE-Bench Lite with 300 instances, TrajSpec boosted Pass@1 from 41.00% to 59.67% with GPT-5-mini and from 54.67% to 64.33% with MiniMax M2.5.
- Ablation studies showed that removing repository-based review or hierarchical evidence representation reduced Pass@1 by up to 11.67%.
Why It Matters
TrajSpec could significantly reduce debugging time and effort for developers by improving AI-driven bug fixing accuracy and reliability.