MARCH scales recurrent memory with content-routed state anchors

Transformers excel at long-context retrieval because their token-level memory grows with context, but this comes at a steep price: quadratic computation during training and a key-value cache that expands linearly during inference. Recurrent alternatives compress all history into a fixed-size state, making decoding efficient but often losing earlier information as new updates overwrite old associations. This forces a painful trade-off between efficiency and recall on retrieval-heavy tasks.

MARCH (Memory-Anchor Routing across Context History), proposed by a multi-institution team, breaks that trade-off. The architecture periodically caches cumulative recurrent-state checkpoints as "state anchors," each associated with a compact, content-conditioned key. At every token, MARCH computes an anchor query and attends over all causally available anchors, aggregating them into the current state. This lets the memory bank grow naturally with context length, offering a controllable balance between historical resolution and memory cost — without abandoning the efficient recurrent computation path.

After standard pretraining, MARCH consistently outperforms multiple linear attention variants across commonsense reasoning, LongBench (a long-context benchmark suite), and in-context retrieval tasks. The results show that content-routed state caching substantially strengthens recurrent long-range memory, giving researchers a practical path toward models that combine Transformer-level retrieval quality with the linear-time efficiency of recurrent architectures.