Predictive coding networks match backpropagation in wide models

A new paper from Francesco Innocenti, El Mehdi Achour, and Rafal Bogacz (University of Oxford) tackles a fundamental question in neuro-inspired AI: can predictive coding (PC) — a biologically plausible learning algorithm that minimizes an energy function locally — scale as effectively as backpropagation (BP)?

The authors rigorously analyze the infinite width and depth limits of predictive coding networks (PCNs). For linear residual networks, they prove that the set of width- and depth-stable feature-learning parameterizations is exactly the same for PC and BP. Moreover, they show that when model width is much larger than depth, the equilibrated PC energy converges to the quadratic BP loss, meaning PC computes identical gradients to BP. Experiments with nonlinear models (convolutional networks and transformers) confirm that the convergence holds as long as activity equilibrium is reached.

This work constrains which parameterizations are scalable for PC while revealing a plausible mechanism for how the brain might implement error backpropagation using only local synaptic updates — a key step toward reconciling deep learning theory with biological realism.