STeP Framework Uses Temporal Logic to Make Robot VLMs Precise and Interpretable

Vision-language-action (VLA) models excel at generalization but struggle with precise instructions involving spatial, temporal, and logical constraints. Researchers from multiple institutions introduce STeP, a hierarchical framework that uses Signal Temporal Logic (STL) as a shared representation connecting high-level language understanding with low-level robot execution. The high-level policy leverages a VLM to decompose natural language instructions into subtasks, generates STL specifications for each subtask, and selects a low-level policy for execution. STL specifications translate language-derived intent into precise, formal constraints that can be mathematically verified.

The framework offers two execution modes: STL-guided model-predictive control (MPC) enforces constraints directly during trajectory optimization, while a monitoring approach checks constraints during execution of learned policies for perceptually complex or contact-rich behaviors. By integrating STL into plan validation, low-level policy selection, subtask monitoring, and replanning, STeP enables language-derived plans to be checked, optimized, and revised at runtime using a common formal structure. The approach was evaluated on a real-world tabletop domain, demonstrating how formal specifications significantly improve the precision, reliability, and interpretability of language-conditioned robot planning compared to purely learned methods.