Echo: Graph-Enhanced Retrieval and Execution Feedback for Issue Reproduction Test Generation

A research team from multiple universities has developed Echo, a novel AI agent designed to automate one of software development's most tedious tasks: creating test cases that reliably reproduce reported bugs. Unlike traditional methods that require developers to manually write tests after understanding complex codebases, Echo uses a graph-enhanced retrieval system that builds connections between code elements and employs automatic query refinement to better understand the bug context. This approach allows the system to generate more accurate test cases from the start.

Echo introduces several practical innovations, including automatic execution of generated test cases—a first-of-its-kind feature that integrates directly into development workflows. The system doesn't just generate tests; it also creates potential patches and uses the patched version to validate whether a test meets the critical "fail-to-pass" criterion, providing actionable feedback for refinement. Unlike previous approaches that sample and rank multiple candidates, Echo generates a single test per issue, offering better cost-performance trade-offs.

On the SWT-Bench Verified dataset, Echo achieved a 66.28% success rate in generating valid bug-reproducing tests, establishing a new state-of-the-art among open-source approaches. The system represents a significant advancement in AI-assisted software engineering, moving beyond simple code generation to more complex reasoning tasks involving execution, validation, and refinement. By automating the bug reproduction process, Echo could dramatically reduce the time developers spend on triage and diagnosis.