Human in the Loop for Fuzz Testing: Literature Review and the Road Ahead
A new 23-page research paper outlines how human experts and LLMs can guide fuzzing to find deep vulnerabilities.
A team of researchers including Jiongchi Yu, Xiaolin Wen, and Sizhe Cheng has published a forward-looking review paper on arXiv, proposing a systematic research agenda to integrate Human-in-the-Loop (HITL) principles with fuzz testing. Fuzzing is a critical automated technique for finding software bugs, but its reliance on automated heuristics often misses deep, complex vulnerabilities. The paper argues that injecting human expert insight into the fuzzing loop—through visualization for interpretability and real-time steering—can dramatically enhance its effectiveness.
The 23-page review surveys existing HITL fuzzing work and outlines a concrete roadmap focused on three pillars: human monitoring of the fuzzing process, human steering to guide exploration toward hard-to-reach code paths, and human-LLM collaboration. A central theme is navigating the new opportunities and challenges posed by Large Language Models (LLMs), questioning how humans can provide actionable knowledge and meta-knowledge efficiently within an intelligent fuzzing system. The authors advocate for a paradigm shift from fully automated fuzzing to interactive, expert-guided systems that combine AI automation with human strategic oversight, aiming to build a next-generation fuzzing ecosystem.
- Proposes a 3-pillar research agenda: Human Monitoring, Human Steering, and Human-LLM Collaboration for fuzz testing.
- Identifies visualization and on-the-fly expert intervention as key to guiding fuzzers toward complex, deep vulnerabilities.
- Highlights the need to define human roles and leverage expert meta-knowledge within new LLM-powered intelligent fuzzing loops.
Why It Matters
This roadmap could lead to more effective security tools that find critical bugs LLMs and automation miss, improving software safety.