New brain-inspired learning rule cuts AI overparameterization by 13x

A new study from researchers Patrick Stricker, Florian Röhrbein, and Andreas Knoblauch introduces a biologically inspired learning rule that addresses two critical problems in modern AI: overparameterization and energy inefficiency. The rule emulates the brain's structural plasticity, dynamically pruning unnecessary synapses and retaining only those essential for the task. Tested on the MNIST classification benchmark, the method naturally prevents overparameterization—which the authors note can bloat standard ANNs by factors of up to 13—while maintaining or surpassing accuracy. Unlike backpropagation, the proposed algorithm does not require manual pre-optimization of network architecture, making it more adaptable and reflective of how biological brains allocate resources.

The implications extend far beyond simple digit recognition. As large language models (LLMs) continue to scale, the environmental and ethical costs of training and running overparameterized networks become increasingly untenable. By eliminating redundant synapses and optimizing synaptic usage, this brain-inspired approach could dramatically reduce energy consumption in future AI systems. The paper also highlights that the learning rule reserves 'space' for new memories, akin to the brain's ability to adapt without catastrophic forgetting. Accepted for the 10th BWHPC Symposium, this work provides a practical framework for building scalable, energy-efficient neural networks that learn more like the human brain.