LLM method synthesizes formal specs from tests alone, no source code needed
Researchers use LLMs to infer Java specs from test traces, bypassing white-box code access.
Researchers have developed an LLM-based approach that synthesizes formal Java Modeling Language (JML) specifications using only test code and dynamic execution traces. The method avoids white-box source code access, reducing intellectual property risks and deployment costs. Candidate specifications are validated locally using bounded model checking, with feedback guiding iterative refinement. Initial results on the SpecGenBench benchmark suggest that tests can guide LLMs toward meaningful JML specifications, while also highlighting checker compatibility and diagnostic feedback as key challenges for reliable refinement.
- Black-box approach: LLM infers JML specs from tests and traces, no source code access needed
- Uses bounded model checking for validation and iterative refinement of candidate specs
- Initial SpecGenBench results show promise, but checker compatibility and feedback quality remain challenges
Why It Matters
Removes IP and cost barriers to formal verification, enabling safe spec inference for proprietary industrial code.