Emergence of Spatial Representation in an Actor-Critic Agent with Hippocampus-Inspired Sequence Generator

A research team led by Xiao-Xiong Lin has published a breakthrough paper accepted at ICLR 2026, demonstrating how a biologically-inspired AI agent can develop spatial navigation capabilities similar to mammalian hippocampus function. Their 'actor-critic agent with hippocampus-inspired sequence generator' proposes that hippocampal place cell sequences arise from intrinsic recurrent circuitry that propagates transient inputs over long horizons, serving as a temporal memory buffer particularly useful when sensory evidence is sparse. This challenges traditional views attributing sequential firing solely to sensory input or planning functions.

The agent was tested on continuous maze navigation using only egocentric visual inputs with extreme sparsity (16 channels, ~2.5% activity). Crucially, it outperformed conventional LSTM-based architectures under these sparse conditions, though not with dense inputs, revealing a strong interaction between representational sparsity and memory architecture. Through reinforcement learning, the model's units developed localized place fields, distance-dependent spatial kernels, and task-dependent remapping—phenomena that align with neurobiological data. The research shows that sparse input synergizes with sequence-generating dynamics, providing both a mechanistic account of biological navigation and a simple yet powerful inductive bias for reinforcement learning agents operating with limited sensory information.