Unifying Dynamical Systems and Graph Theory to Mechanistically Understand Computation in Neural Networks

In neural systems, structural and functional connectivity often diverge, complicating the understanding of how computation emerges from connectivity. This paper by Sharma et al. shows that by modeling recurrent neural networks (RNNs) as graphs and analyzing multi-hop pathways between input and output units, one can recover the network's spatial and temporal function. Decomposing pathways by hop length reveals how the network temporally routes information. This reframes regularization: standard penalties like L1 only constrain single-hop structure, not the multi-hop pathways that support computation.

To address this, the authors introduce resolvent-RNNs (R-RNNs), which constrain multi-hop pathways and induce temporal sparsity beyond that achieved by L1. R-RNNs achieve improved performance by matching the task structure, even with sparse signals. They also exhibit stronger sparsity-function alignment, reflected in increased robustness under strong regularization. These results identify multi-hop communication as a key principle linking structure to function in recurrent networks, suggesting that sparsity should be defined over functional pathways rather than individual parameters.