Process Sidecars enable revocable memory in safety-trained LLMs

A new paper by John Sweeney tackles a critical flaw in LLM memory editing: when models undergo sequential adaptation (skill learning, private memory injection, then safety training), revoking the memory after the safety phase is not the same as simply subtracting the memory update. The issue is that safety optimizers warp the memory direction, making standard task arithmetic leave first-order residual errors. Sweeney proposes "process sidecars" — a family of linear edits parameterized by two coefficients that leverage process information about the future safety training trajectory. Specifically, the edit reuses the already-trained model checkpoint and computes one extra safety trace to approximate the transported memory direction via a centered secant of the AdamW process. The paper proves two things: first, that the exact sidecar recovers the counterfactual safety-only oracle up to second order; second, that this process information is necessary — no scalar task arithmetic edit can match it. Empirical validation across three models shows that the validation-selected 2D edit improves held-out refusal closure over naive task arithmetic in all trials, and over the simpler gamma=lambda subfamily in all paired comparisons. The work provides a practical method for building revocable memories in safety-critical LLM deployments.

The paper includes 23 pages, 2 figures, and 6 tables, with code and data linked on arXiv. The approach is model-agnostic but evaluated on standard open-weight models.