Reprodgen Framework Turns Q&A Discussions into Executable Buggy and Fixed Code

Reproducing data science code from informal Q&A forums remains a major challenge due to ambiguous specs, missing dependencies, and unstructured discussions. In response, researchers Ragib Shahariar Ayon, Mohammad Wardat, and Shibbir Ahmed present Reprodgen—a large language model (LLM) based framework that automatically transforms forum questions and answers into executable buggy and patched code pairs. The system builds structured representations of Code Intent (CI), Functional Requirements (FR), and a Structured Chain of Thought (SCoT), then iteratively refines code using an LLM-based reviewer until it is both executable and semantically consistent with the original discussion.

Reprodgen was evaluated on Stack Overflow and GitHub Issues across seven major data science libraries—including pandas, numpy, and scikit-learn—and benchmarked against a human-validated corpus of runnable buggy and patched programs. The pipeline uses LLMs for semantic assessment while verifying executability through actual execution. Results demonstrate reliable replication with clear differences in model performance. The paper has been accepted at the 37th IEEE International Symposium on Software Reliability Engineering (ISSRE 2026), underscoring its significance for automated debugging and verification in data science workflows.