LISA framework uses LLMs to catch functional bugs without crashes

A team of researchers (Ruogu Yang, Yifeng He, Yundi Xu, Yuqing Wei, Hao Chen) has published a paper introducing LISA, a novel LLM-based invariant testing framework for detecting functional bugs in software libraries. Traditional methods like heuristic test generation suffer from low usability because they can't reason about program semantics, while fuzzing tools like OSS-Fuzz often miss functional bugs that don't cause crashes. LISA overcomes these limitations by leveraging large language models to iteratively generate API sequences and program invariants, guided by API n-gram feedback. This approach enables LISA to understand the intended semantics of APIs and uncover bugs that would otherwise go unnoticed.

In experiments, LISA demonstrated higher bug-detection rates and competitive code coverage compared to both traditional fuzzing techniques and prior LLM-based test generation approaches. Each finding is reported as a high-confidence bug candidate, reducing developer overhead for verification. The paper has been accepted for publication at ISSRE 2026. This work represents a significant step forward in automated software testing, particularly for finding functional bugs that are invisible to crash-based fuzzers. By combining the reasoning capabilities of LLMs with systematic invariant testing, LISA offers a practical tool for improving software reliability in production environments.