New method scales spiking neural networks to thousands of GPUs

In a new paper published in Neuromorphic Computing and Engineering, Bruno Golosio and a team of 12 researchers introduce a scalable method for constructing spiking neural networks (SNNs) on multi-GPU clusters. Inspired by the human cerebral cortex—a sparsely connected network of roughly 10^10 neurons, each forming 10^3–10^4 synapses and communicating via electrical spikes—the method addresses the challenge of simulating such complex systems at scale. By leveraging the Message Passing Interface (MPI), each process builds its local connectivity and prepares data structures for efficient spike exchange across the cluster during state propagation. This allows the simulation to handle millions of neurons and billions of synapses across thousands of GPUs.

The team demonstrated scaling performance using two distinct cortical models: one relying on point-to-point communication and another on collective communication. Both approaches show strong scalability, paving the way for exascale-level simulations. The work builds on previous efforts in computational neuroscience but introduces a novel network construction phase that minimizes memory overhead and communication bottlenecks. The method is designed to run on the next generation of high-performance computing clusters, potentially enabling researchers to model brain-scale neural activity with unprecedented detail. The paper (arXiv:2512.09502v2) is published in the special issue of Neuromorphic Computing and Engineering, Volume 6, Number 2.