GapForge uses LLMs to find compiler bugs by targeting coverage gaps

Compiler fuzzing is a key technique for finding bugs in critical infrastructure like GCC and LLVM, but most existing methods struggle with long-tail coverage gaps — hard-to-reach code regions that remain untested across releases. GapForge, a new technique from Mingxuan Zhu and colleagues, tackles this head-on by using large language models to reason about coverage gaps at the region level. Instead of generating random programs or relying on whole-file summaries, GapForge proceeds in three steps: it prioritizes undercovered files using a coverage-driven scoring system, then pairs uncovered line spans with their enclosing covered context and performs path-difference analysis to infer what program structures and compiler flags are needed to reach the gap. Finally, it synthesizes prompts from these requirements, iteratively refining them based on coverage feedback from previous runs.

Tested on GCC 14.3.0 and LLVM 19.1.0, GapForge significantly outperformed eight state-of-the-art techniques including WhiteFox, a white-box fuzzer. Within 72 hours, it achieved 68.13% coverage on GCC core modules and 69.11% on LLVM — surpassing WhiteFox by 24,736 and 19,798 additional lines respectively. More importantly, GapForge uncovered 12 real-world compiler failures: 5 in GCC and 7 in LLVM, comprising 8 crashes and 4 miscompilations. The paper demonstrates how LLM-based test generation, when guided by structural coverage analysis, can dramatically improve compiler testing and harden the software supply chain.