MetaNCA learns to generate neural network weights without backpropagation
A new AI framework lets networks self-organize weights purely through local interactions, no backprop needed.
MetaNCA draws inspiration from biological self-organization, where local interactions drive complex emergent behavior. Traditional neural networks rely on global backpropagation, which is costly and biologically implausible. The MetaNCA framework instead learns a rule network (the MetaNCA) that iteratively updates the weights of a task network using only local information on the computation graph. A key innovation is the Weight Transformer architecture, which uses linear attention to aggregate signals from neighboring weights and hidden states, enabling efficient local updates.
Tested on MNIST and CIFAR-100, MetaNCA successfully generates weights for feedforward MLPs, convolutional networks (CNNs), and residual networks (ResNets), scaling to networks with 2 million parameters. Crucially, the trained rule network generalizes to entirely new architectures not seen during meta-training, with architectural diversity in the training phase strengthening this capability. This work, accepted at ALIFE 2026, opens the door to more adaptive, efficient, and biologically-inspired learning systems that could reduce reliance on backpropagation in future AI.
- Introduces Weight Transformer with linear attention for local weight updates without backpropagation
- Scales to 2 million parameters across MLPs, CNNs, and ResNets on MNIST and CIFAR-100
- Generalizes to architectures unseen during meta-training, especially when trained on diverse architectures
Why It Matters
Could enable adaptive, biologically-plausible AI that learns without costly global backpropagation.