Continual-learning rules shape representational drift, new study finds

A new paper from Yikai Si and Shanshan Qin, posted on arXiv (2608.16141), tackles a central puzzle in lifelong learning: how can neural networks preserve old knowledge while their internal codes keep shifting? The researchers trained convolutional networks on sequential image classification tasks and recurrent networks on sequences of cognitive tasks, then tracked fixed probe representations throughout learning. Their goal was to see whether different continual-learning mechanisms produce distinct patterns of representational drift—the slow change in how familiar stimuli are encoded over days or weeks.

Their results show that experience replay—a classic method for preventing catastrophic forgetting—preserved earlier task performance in both architectures, but representations drifted progressively with the number of intervening tasks. The drift was structured: later visual-processing stages and recurrent units' temporal tuning were especially labile, while coarse class organization and task-relevant temporal structure remained stable. In contrast, an algorithm that strongly anchored weights nearly froze representations, and directly anchoring an old representation during replay suppressed drift but impaired acquisition of subsequent tasks. This demonstrates that drift is not random noise but a reflection of the stability-plasticity tradeoff: the same mechanism that protects old knowledge shapes how much representations move, and suppressing drift entirely can restrict future learning. The authors suggest drift could serve as an observable signature of the constraints that enable continual learning in both brains and machines—a potentially useful diagnostic for AI systems designed for lifelong learning.