Störk's IGFA method achieves lossless retention in continual learning without replay

Julius Störk's new paper, "Interference and Retention in Continual Learning," rethinks how neural networks forget. Instead of relying on post-hoc mechanisms like replay, elastic regularization, or distillation, Störk models forgetting directly as interference between tasks. In the frozen-feature regime, learning a new task induces interference energy on old tasks, which can be exactly measured. For deep networks, this same quantity is recovered via path-averaged curvature with minimal extra forward passes. This geometric insight leads to a key finding: when task supports are disjoint, forgetting can be eliminated structurally. When they overlap in conflicting directions, a non-zero distortion floor is unavoidable, but the cost can be moved from irreversible forgetting to deferred, recoverable plasticity.

From this analysis, Störk derives Interference-Gated Functional Allocation (IGFA), a replay-free and Fisher-free method. IGFA shares gradient directions when tasks align and protects them when they conflict. Across benchmarks spanning 41 pages and 21 figures, IGFA achieves lossless retention on structurally separable tasks. On dissimilar-task streams, it matches the strongest replay-free structural baselines; on similar tasks, it outperforms unconditional projection by preserving transfer. This approach avoids the memory and computation overhead of replay buffers and Fisher approximations, making it practical for resource-constrained continual learning scenarios.