agrepl framework enables 100% deterministic replay for debugging AI agents
Open-source CLI tool achieves replay fidelity F=1.0 and cuts per-step latency by 98.3%
AI agent systems that combine LLMs with external tools and APIs are notoriously non-deterministic due to sampling variance, API state changes, and execution noise—making it nearly impossible to faithfully reproduce a prior run. Existing observability tools capture logs but cannot recreate isolated executions. To solve this, Rasheed Mudasiru presents agrepl, a record/replay framework implemented in Go that ships as a single static binary under the MIT license.
agrepl intercepts all external interactions at the transport layer using a man-in-the-middle (MITM) proxy, serializing them as structured execution traces. Traces are replayed in a strictly isolated environment with zero outbound network access, ensuring complete determinism. The paper formalizes a request-key matching function K(s) and proves the determinism invariant, while a noise-aware diff algorithm classifies HTTP header divergence into signal and noise tiers. In empirical evaluations across five workloads (n=250), agrepl achieved replay fidelity F=1.0 and a median per-step latency reduction of 98.3%.
- MITM proxy at transport layer captures all external LLM and API interactions without code modification
- Zero outbound network during replay guarantees isolation and prevents state pollution
- Noise-aware diff algorithm distinguishes critical HTTP headers from irrelevant variations
Why It Matters
Developers can now reliably reproduce and debug AI agent failures, accelerating iteration and reducing debugging time by orders of magnitude.