ReDig framework uses runtime feedback to refine LLM-based test generation

LLM-based directed test input generation has shown promise for creating test inputs that reach specific code lines, but it remains unreliable due to limited code context and the inherent unpredictability of LLM inference. To address this, researchers Narin Han and Shin Hong from (presumably) academia propose ReDig, a runtime feedback-guided refinement framework that adds a control loop around any LLM-based generation technique. Instead of accepting a failed test script, ReDig collects runtime values observed during target-missing executions, uses them to diagnose why the script failed to reach the target, and refines the generation prompt with that feedback. This grounds the LLM's next attempt in concrete observed behavior rather than pure inference.

In case studies on Poppler and Libsndfile—two widely used C libraries for PDF rendering and audio file handling—ReDig demonstrated that runtime value feedback can effectively explain prior failures and guide the LLM to produce more precise test scripts in subsequent iterations. The framework works generically with existing LLM-based generators, making it a lightweight enhancement rather than a replacement. While the paper doesn't provide quantitative benchmark numbers, the qualitative results show clear diagnostic value: ReDig transforms vague 'didn't reach target' failures into actionable insights about variable states and execution paths. For software engineers, this could mean more reliable automated test generation for complex, real-world codebases—potentially reducing the manual effort needed to write target-reaching tests.