SpecSyn: LLM-based Synthesis and Refinement of Formal Specifications for Real-world Program Verification

Program verification is critical for ensuring software correctness, but writing the necessary formal specifications is notoriously time-consuming. A team of researchers from multiple institutions has introduced SpecSyn, a novel LLM-based approach that automates this process. SpecSyn works by first decomposing large input programs into smaller, manageable segments, then applying an iterative generation process powered by large language models. What sets SpecSyn apart is its refinement mechanism: it uses semantic-non-equivalent program mutations and variant discrimination to evaluate and enhance the strength of generated specifications, ensuring they are precise enough for real-world verification tasks.

In extensive experiments, SpecSyn demonstrated impressive performance, maintaining high precision above 90% and outstanding recall over 75%, significantly outperforming existing LLM-based specification generation methods. The tool successfully handled 1,071 out of 1,365 target properties in open-source programs, proving its applicability to real-world software verification. This breakthrough could dramatically reduce the manual effort required for formal verification, making it more accessible for developers working on complex, safety-critical systems.