Yohei Nakajima's Regimes uses ActiveGraph to create auditable AI self-improvement loops

Yohei Nakajima's new paper, "Regimes: An Auditable, Held-Out-Gated Improvement Loop Demonstrated on LongMemEval with ActiveGraph," tackles a fundamental trust problem in autonomous AI systems. Most improvement loops bolt on external scaffolding that fails to log failures, replay diagnoses, or track promotion decisions in the agent's own history. Nakajima's solution is an event-sourced agent runtime called ActiveGraph. Here, the agent's state is a deterministic projection of an append-only event log. Every failure is recorded, every run replays exactly, and candidate patches scope to typed pipeline seams with auditable gates. The loop diagnoses failures, proposes repairs, and promotes them only after static checks, sandbox execution, in-sample evaluation, and held-out validation. The process is target-agnostic, running the same control flow across different tasks.

On LongMemEval-S—a long-context evaluation benchmark—Regimes uncovered a surprising failure mode: the dominant issue is not retrieval but reconciliation. The evidence is already in the assembled context, yet the reader model answers incorrectly. Across five seeded held-out splits, Regimes discovered reader-prompt repairs that boosted final held-out accuracy by +0.05 to +0.10 in four splits and +0.01 in one over-promotion split. Two splits were individually significant. The durable contributions of the paper include ActiveGraph as an auditable substrate for controlled improvement loops, the held-out-gated loop paradigm, a failure-regime taxonomy that routes each error to a specific pipeline location, and the "prompt-as-discovery-probe" hypothesis. The work makes autonomous AI improvement auditable, replayable, and trustworthy—critical for real-world deployment.