Meta-Representational Predictive Coding: Neuroscience-inspired SSL bypasses backprop
No generative model, no backprop – MPC learns representations through active sensory glimpsing.
Self-supervised learning (SSL) has become a cornerstone of modern AI, but most methods rely on backpropagation – a biologically implausible credit assignment mechanism – and feedforward inference. Predictive coding offers a more brain-like alternative, but existing variants either require learning a costly generative model of raw input (unsupervised) or human labels (supervised). In a new paper, Alexander Ororbia (RIT), Karl Friston (UCL), and Rajesh Rao (University of Washington) propose Meta-Representational Predictive Coding (MPC), a framework they term 'neuroscience-informed self-supervised learning' (NeuroSSL). MPC sidesteps generative modeling entirely by learning to predict representations across parallel processing streams, resulting in an encoder-only architecture that is both efficient and biologically plausible.
Crucially, MPC incorporates active inference in the form of 'sensory glimpsing' – the model learns to make sequential decisions about which informative portions of its sensorium to sample, driving the representational dynamics. This saccade-like behavior mirrors how biological vision systems actively explore scenes. The authors demonstrate that MPC achieves strong zero-shot generalization on natural image benchmarks without needing pixel-level reconstruction or backpropagated gradients. The work is a significant step toward merging neuroscience principles with practical self-supervised learning, potentially reducing the energy and data demands of current SSL models.
- MPC learns encoder-only representations by predicting across parallel streams, eliminating the need for generative models or pixel-level targets.
- Uses active inference (sensory glimpsing) where the model decides which parts of the input to sample, mimicking biological saccades.
- Demonstrates zero-shot generalization on natural images without backpropagation, offering a more brain-like alternative to contrastive SSL.
Why It Matters
Enables more efficient, biologically plausible self-supervised learning that could reduce compute and data requirements for vision.