Sequential AI edits fail: order matters, knowledge resurfaces
Cutting AI memories in sequence breaks the model, with 61–71% divergence.
A new paper from Ferdinand Schessl (arXiv:2607.24805) challenges the foundational assumption of AI Engram—a method that extracts and combines concept-specific memory traces in neural networks. The original paper hypothesized that edited models live on a "commutative manifold" where integrating edits A and B reaches the same equilibrium regardless of order. Schessl runs sequential edit tests on the authors' own implementation at the best-case edit strength (TOFU alpha=0.6) across three model charges (two vendors, two architectures).
Four key findings replicate across all tests: (1) zero-shot composition and sequential re-calibrated editing diverge by 61–71% of edit magnitude; (2) cut order is not interchangeable—in one case, the order of deleting two Paris landmarks decided whether an unrelated third concept survived; (3) the method's own sufficient statistics (layer-input covariances) drift monotonically with each new cut in every surviving concept; (4) erased knowledge partially returns under subsequent unrelated edits. This falsifies the commutative-manifold hypothesis for sequential editing, with profound implications for unlearning-as-compliance: a model certified free of certain knowledge today may not remain so after its next edit.
- Sequential edits cause 61–71% divergence from expected zero-shot composition results.
- Order of deletion matters: cutting two Paris landmarks in different orders can kill an uninvolved third concept.
- Erased knowledge partially resurfaces after subsequent unrelated edits, threatening compliance guarantees.
Why It Matters
AI unlearning certifications are fragile—sequential edits break memory isolation, challenging compliance in regulated models.