Ikemoto's noise neural nets eliminate weight transport with forward-only BP

A new paper by Shuhei Ikemoto introduces a method to reconstruct backpropagation in neural networks using only forward-pass fluctuations, solving the long-standing weight transport problem. Traditional backpropagation requires a symmetric reverse path with transposed weight matrices, which is biologically implausible and difficult to implement in neuromorphic hardware. Forward-only alternatives usually sacrifice stability or accuracy. Ikemoto's Noise-modulated Neural Network (NNN) instead leverages noise as a computational resource: a weight mirror estimates each weight matrix from the covariance between a previous-layer unit's output and the next-layer unit's input. Combined with local differential estimation inside the units, the output error is propagated recursively along the computational graph without any transposed-weight readout or backward data path.

The resulting gradient is empirically near-unbiased, and when combined with local per-weight Adam updates, it matches the final accuracy of standard backpropagation on simple regression tasks. The approach works with uniformly distributed noise, reducing local operations to polynomials and comparators, making the entire learning rule well-suited for digital circuits. This work positions noise not as a nuisance but as a resource for both inference and learning, potentially enabling biologically plausible and hardware-efficient neural network training.