First ImageNet-scale PCN trained with Equilibrium Propagation hits near-backprop accuracy
A 10-layer predictive coding network achieves 13.23% top-5 error on full ImageNet.
Equilibrium Propagation (EP) is a physics-inspired training framework for energy-based models like continuous Hopfield networks and nonlinear resistive networks, but it has been limited to small-scale problems. Similarly, Predictive Coding Networks (PCNs), rooted in computational neuroscience, have never been demonstrated at large scale. In a new paper, Tugdual Kerjan, Rasmus Høier, and Benjamin Scellier introduce an EP-based training method for PCNs that uses the centered variant of EP combined with a novel equilibration scheme.
Applying this approach, they train a 10-layer convolutional PCN (VGG10) on full-size ImageNet. The model achieves a 13.23% top-5 test error rate, closely approaching the 12.2% baseline achieved by standard backpropagation. This marks the first time both PCNs and EP-based training have been scaled to ImageNet, a significant milestone. The results suggest that the primary challenges in scaling EP to other physical systems may stem from the computational properties of those systems rather than inherent limitations of the EP framework itself.
- Developed a novel EP-based training method combining centered EP with a new equilibration scheme for PCNs.
- Trained a 10-layer convolutional PCN (VGG10) on full-size ImageNet, achieving 13.23% top-5 error (vs 12.2% backprop baseline).
- First-ever demonstration of both Predictive Coding Networks and Equilibrium Propagation at ImageNet scale.
Why It Matters
This work scales biologically plausible learning to large tasks, challenging backpropagation's dominance and opening doors for physics-based training.