New AI framework uses 8 LLMs to auto-generate code docs, finds 42% gap

High-quality source code documentation is often neglected but critical for reliability in domains like healthcare. Researchers from Tunisia (Ikbel Ghrab et al.) presented an AI-powered framework that automates documentation generation from code and repositories using eight state-of-the-art Large Language Models (LLMs), including GPT, Gemini, Qwen, and LLaMA variants. Built on the PocketFlow orchestration framework, the system applies modular pipelines and advanced prompt engineering to produce structured, context-aware documentation. The experiments were conducted on an open-source medical physics library, demonstrating the framework's ability to generate comprehensive documentation automatically.

To ensure quality and guide model selection, the team introduced a MultiLLMasJudges evaluation framework where four independent LLMs assess outputs across nine criteria, such as Completeness, Clarity, and Faithfulness. Results showed a 42% performance gap between the top and bottom models, highlighting significant differences in LLM capabilities for this task. By combining diverse model outputs, optimized prompting, and rigorous multi-judge evaluation, the approach enhances documentation quality and reduces manual effort—especially in safety-critical healthcare software. This work was presented at the ICAHS 2025 conference and published on arXiv.