Agent frameworks fail 'stop' controls – SOUNDGATE enforces with 12k/sec admissions
215 out of 1,200 runs executed effects despite approval pause – here's the fix.
Deep Dive
New research by Sajjad Khan tests six widely used open-source LLM-agent frameworks and finds that none actually enforce barrier semantics for control primitives like approval gates, cancellation, and timeouts. A recurring 'sibling leak' allows side effects to execute during a pause. Frontier models produce leak-triggering plans at up to 14% rate. The proposed solution, SOUNDGATE, is a Rust-based external gate that blocks all violations at about 1 ms admission per write and handles 12k-26k durable admissions per second.
Key Points
- All six tested open-source agent frameworks fail to enforce barrier semantics for approval gates, cancellations, and timeouts.
- A 'sibling leak' lets side effects execute during a pause – frontier models produce trigger patterns up to 14% of the time.
- SOUNDGATE, a Rust-based external gate, blocks all violations with ~1ms admission latency and 12k-26k admissions/sec throughput.
Why It Matters
AI agents could execute unintended actions despite cancellation or timeout, risking data corruption and security breaches in production.