Self-supervised local learning rules crack hierarchical data without backprop

A new study from Ariane Delrocq, Wu S. Zihan, Guillaume Bellec, and Wulfram Gerstner tackles a fundamental question in neuroscience and machine learning: How can the brain learn abstract hierarchical representations from high-dimensional sensory input using only local plasticity rules? The authors use the Random Hierarchy Model (RHM)—a synthetic dataset designed to test deep neural networks' ability to learn intrinsic hierarchies—to evaluate two families of biologically plausible learning algorithms.

The first family uses direct feedback signals to approximate error propagation from the output layer (similar to feedback alignment). The second family employs layerwise self-supervised loss functions—either contrastive (e.g., SimCLR-style) or non-contrastive (e.g., Barlow Twins-style)—that never explicitly approximate output errors. Results show that all type 1 rules fail on RHM tasks, a failure traced to input-specific nonlinearities ('masking') that full backpropagation implements and that are essential for complex hierarchical learning. However, type 2 self-supervised local rules succeed, matching the data efficiency of supervised backprop while remaining compatible with known rules of synaptic plasticity in the cortex. This suggests the brain might use local self-supervised objectives rather than approximating global error signals.