Structured noise fields enable multi-function neural networks with spatial subnetwork selection

A new paper from Shuhei Ikemoto and Fabio DallaLibera turns a common assumption on its head: noise in neural networks doesn't have to be a nuisance. Instead, they propose using structured noise fields to selectively activate different subnetworks within a single network, enabling it to perform multiple distinct functions without retraining. The core innovation is the crossing activation function, which comes in three implementation variants—sample-level, statistical-level, and analytical-level—that allow flexible parameter reuse across different computational contexts.

By introducing a 'virtual noise field'—a continuous auxiliary space that generates spatially structured network noise—the authors demonstrate how partially overlapping subnetworks can be activated. In one-dimensional function approximation tasks, they show that multiple functions can coexist in a single network when each function is assigned a different location in the noise field. Crucially, memory capacity improves when the spatial arrangement of noise fields reflects the proximity relationships among the learned functions. Mismatches degrade performance, highlighting that structured noise serves not just as a perturbation but as a topology-defining factor for functional subnetwork selection. This work, published on arXiv (2606.24588), points toward more efficient, compact neural architectures capable of dynamic task allocation.