Urysohn Ladder uses metric contraction for scalable continual learning

Continual learning systems face a fundamental geometric obstacle: as experience accumulates on a fixed-capacity manifold, covering numbers grow linearly with time, eventually forcing representational overlap and catastrophic interference. Prevailing approaches attack this problem by — expansion, projecting into higher-dimensional spaces via kernels, overparameterization, or replay. Xin Li’s new paper argues the solution is the opposite: contraction.

The proposed Urysohn Ladder formalizes abstraction as a hierarchy of quotient maps that recursively collapse validated metric neighborhoods into compact tokens. Each collapsed token acts as a “shortcut” — a region of extreme metric contraction that bridges distant experiences, much like a wormhole in the representational manifold. Li establishes four theoretical guarantees: separability (nonlinearly entangled structure becomes linearly separable at each quotient level), bounded capacity (covering numbers remain O(1) per level, independent of stream length), stability (parity-partitioned flow and scaffold subspaces enable unbounded plasticity without catastrophic interference), and scalability (inference cost scales with quotient distance, not ambient distance). These claims are validated empirically with pretrained models and real-world datasets, demonstrating the potential of the Urysohn Ladder for scalable continual learning via scaffold amortization.