Orthogonal Weight Modification Enhances Learning Scalability and Convergence Efficiency without Gradient Backpropagation

Researchers Guoqing Ma and Shan Yu have introduced LOCO (LOw-rank Cluster Orthogonal), a novel weight modification algorithm that enables neural network training without relying on computationally expensive backpropagation (BP). Inspired by neural representations in the brain, LOCO uses a perturbation-based approach where it applies low-rank, orthogonal modifications to network weights. This method addresses critical efficiency and scalability challenges that have plagued previous non-BP alternatives, allowing it to train significantly deeper networks than previously possible with brain-inspired algorithms.

The technical breakthrough lies in LOCO's constraint of orthogonality, which limits variance in gradient estimates from node perturbation and dramatically improves convergence efficiency. In extensive evaluations, LOCO demonstrated the capability to locally train spiking neural networks with more than 10 layers—the deepest achieved to date without backpropagation—while showing superior continual learning ability and task performance compared to other non-BP methods. Most notably, LOCO requires only O(1) parallel time complexity for weight updates, compared to the O(n) complexity of traditional BP methods. This represents a major step toward enabling high-performance, real-time lifelong learning on emerging neuromorphic computing systems that mimic biological brain architecture.