Assessing Reproducibility in Evolutionary Computation: A Case Study using Human- and LLM-based Assessment

Researchers analyzed ten years of papers from a major AI conference to see if their results could be reproduced. They found that papers scored only 62% on average for providing necessary details, and just over a third shared supporting materials. The team also created an AI tool, RECAP, which successfully automated this assessment, matching human judgment 67% of the time. This highlights a significant reporting gap but shows automated checks are a viable solution.