Chaotic RNNs enable smooth neural codes via intrinsic regularization
How chaotic neural networks maintain stable population codes while being intrinsically chaotic.
Cortical circuits operate in an intrinsically chaotic regime where tiny input changes cause divergent responses. Yet population codes in the brain vary smoothly with stimuli, forming coherent representational manifolds. This paradox has puzzled neuroscientists. Now, a team from Forschungszentrum Jülich (Bauer, Keup, Kadmon, Helias) presents a unified framework in arXiv:2606.04426 that resolves this tension.
Combining kernel methods with dynamical mean-field theory, the researchers show that chaotic dynamics in recurrent neural networks create local roughness—sharp distortions at small scales—while preserving global smoothness across larger stimulus variations. This structural property acts as an intrinsic regularizer, enhancing generalization without sacrificing expressivity. Crucially, the model produces power-law spectral signatures that closely match experimental cortical recordings, providing a direct link between network dynamics and observed neural activity.
The work has profound implications for both neuroscience and AI. It explains why chaotic spiking neural networks can sustain differentiable population codes, and offers new design principles for building more robust and generalizable artificial neural networks that naturally balance chaos and stability.
- Chaotic dynamics produce local roughness but global smoothness, acting as an intrinsic regularizer in RNNs.
- Framework uses kernel methods and dynamical mean-field theory to link microscopic chaos to macroscopic neural geometry.
- Power-law spectral signatures from the model closely match experimental cortical recordings, validating the theory.
Why It Matters
Explains how chaotic neural networks sustain smooth population codes, with direct impact on AI regularization and neuroscience.