ClarifySTL: An Interactive LLM Agent Framework for STL Transformation through Requirements Clarification
Turns ambiguous natural language into precise Signal Temporal Logic with interactive clarification.
Formal specifications like Signal Temporal Logic (STL) are critical for cyber-physical systems, but translating vague or ambiguous natural language requirements into STL remains a major challenge. Existing LLM-based approaches often fail when requirements contain underspecified terms or multiple interpretations.
To address this, researchers propose ClarifySTL, an interactive LLM agent framework that first detects vague expressions in a requirement, then generates targeted queries to guide users in supplementing missing details. If ambiguity persists, it formulates focused clarification queries and updates the requirement based on user feedback before finally transforming it into STL using LLMs. The 32-page paper (arXiv:2605.01209) evaluates ClarifySTL on DeepSTL, STL-DivEn, and a newly introduced AmbiEval benchmark designed to test vagueness and ambiguity handling. Results show the framework effectively reduces ambiguity while keeping user burden low.
- Detects vague expressions and generates targeted clarification queries before STL transformation.
- Introduces AmbiEval benchmark specifically for assessing vagueness and ambiguity handling in STL.
- Evaluated on DeepSTL and STL-DivEn datasets; outperforms baseline LLM approaches without clarification.
Why It Matters
Automates formal verification for CPS, reducing mis-specifications caused by ambiguous requirements in safety-critical systems.