Developer Tools

SIGIL compiles AI agent skills into typed harnesses, hitting 86% step compliance

AI agents skip 44% of steps in prose skills; SIGIL's compiled harnesses reach 86% compliance.

Deep Dive

AI agents increasingly gain capability from prose skill files—plain-language procedures loaded into context and executed via a tool-calling loop. The problem: these skills are described to the runtime but never encoded in it, so the model re-derives control flow on every run and frequently skips mandated checks. Across 30 skills and two model generations, a prose-driven agent performs only 56% of the steps its own skill mandates, while still producing artifacts that pass output validation. That means even when outputs look correct, the agent isn't actually following its instructions.

SIGIL, from the same research group behind Clinc, solves this with Skill Compilation: it takes a prose skill and compiles it into a typed executable harness. At the core is AG-IR, an agentic intermediate representation that cleanly separates model-owned cognition (reasoning, planning) from code-owned mechanism (enforced verification, control flow). The results are striking—compiled harnesses execute 86% of mandated steps, complete the full procedure 2.3x more often, and require 0.58x the tokens. Even better, the guarantee is model-independent: the 86% holds across two model generations, while prose compliance swings from 56% to 68%. Developers keep the easy authoring surface of prose, but gain the reliability of code-level enforcement.

Key Points
  • Prose-based agents only execute 56% of steps their own skills mandate across 30 skills and two model generations.
  • SIGIL-compiled harnesses hit 86% step compliance, complete full procedures 2.3x more often, and use 42% fewer tokens.
  • SIGIL's AG-IR intermediate representation separates model cognition from code-owned mechanism, and the 86% guarantee holds across models.

Why It Matters

Reliable, instruction-following agents are critical for autonomous workflows; SIGIL narrows the gap between prose authoring and code-level guarantees.

📬 Get the top 10 AI stories daily