New SDM Encoding Boosts Memory Robustness by 13.4% for Continual Learning

Large language models struggle with continual learning, spurring interest in Sparse Distributed Memory (SDM) as an explicit online episodic memory. The recent CALM architecture (Nechesov & Ruponen, 2025) left its threshold-binary encoder as an open design question. Joy Bose's paper systematically evaluates rank-order N-of-M encoding (Furber et al., 2007) as a drop-in alternative. The work first validates a faithful reimplementation, showing exact equivalence between WheelSDM and RankOrderSDM (cosine similarity 1.0000 across 10 seeds) and reproducing RDLIF neuron divergence under interference. Capacity experiments then demonstrate RankOrderSDM outperforming StandardSDM by 13.4 percentage points at saturation in a scaled configuration and by 0.8 pp at the original published scale.

To isolate the source of robustness, the author runs BER robustness experiments that disentangle representation from learning effects. The key finding: the large robustness gain arises primarily from the interaction of rank-order encoding with MAX-Hebbian learning, while the encoder alone provides only a small advantage under matched learning conditions. Tests on GloVe-100 embeddings confirm a small but consistent encoding benefit on real structured data, whereas sentence embeddings exhibit a ceiling effect at low memory load. A secondary energy analysis shows idealized rank-order encoding requires half the component-level encoding energy of SpikingMamba's SI-LIF neurons at four-bit precision, though decoder costs dominate overall system energy. The paper offers practical guidance for memory-augmented AI architectures like CALM.