LLMLOOP: Improving LLM-Generated Code and Tests through Automated Iterative Feedback Loops

A research team including Ravin Ravi, Dylan Bradshaw, and Valerio Terragni has introduced LLMLOOP, a novel framework designed to tackle the persistent quality issues in code generated by Large Language Models (LLMs). While models like GPT-4 and Claude excel at initial code generation, the output often contains compilation errors, logic bugs, or poor test coverage, forcing developers into tedious manual review cycles. LLMLOOP automates this refinement process through a structured series of five iterative feedback loops. These loops systematically address compilation errors, static analysis warnings, failing test cases, and crucially, employ mutation analysis to assess and improve the quality of the generated test suites themselves. This ensures the final code is not just syntactically correct but also accompanied by robust, validating tests.

The framework was rigorously evaluated using the HUMANEVAL-X benchmark, a standard for assessing code generation across multiple programming languages. The results demonstrate that LLMLOOP effectively elevates the quality of raw LLM output, transforming buggy or incomplete drafts into production-ready code and comprehensive test suites. By automating the iterative 'fix-and-verify' cycle that developers typically perform manually, LLMLOOP promises to drastically reduce wasted effort and increase trust in AI-assisted programming. The work is accepted for publication at the IEEE International Conference on Software Maintenance and Evolution (ICSME 2025), highlighting its practical significance for the software engineering community. This tool represents a shift from using LLMs as mere code drafters to integrating them into a reliable, automated software development pipeline.