Free energy principle yields self-orthogonalizing attractor networks

Tamas Spisak and Karl Friston have published a paper in Neurocomputing (2026) deriving attractor neural networks from first principles using the free energy principle. By applying a universal partitioning of random dynamical systems, they show that attractor dynamics emerge without the need for explicitly imposed learning or inference rules. The resulting networks perform a collective, multi-level Bayesian active inference where attractors encode prior beliefs, inference integrates sensory data into posterior beliefs, and learning fine-tunes couplings to minimize long-term surprise. Crucially, the networks favor approximately orthogonalized attractor representations, a consequence of simultaneously optimizing predictive accuracy and model complexity. These orthogonal attractors efficiently span the input subspace, boosting mutual information between hidden causes and observable effects and improving generalization.

The paper also reveals a natural generalization of conventional Boltzmann Machines. While random data presentation leads to symmetric and sparse couplings, sequential data fosters asymmetric couplings and non-equilibrium steady-state dynamics. This provides a biologically plausible framework for learning and inference that may inspire more efficient AI architectures and deepen our understanding of neural computation in the brain. The findings offer a unifying perspective on self-organizing neural networks, bridging neuroscience and machine learning.