Collapse or Preserve: Data-Dependent Temporal Aggregation for Spiking Neural Network Acceleration

A new research paper by Jiahao Qin challenges a fundamental assumption about Spiking Neural Networks (SNNs), which are brain-inspired models prized for their energy efficiency. The widespread belief that the 'sparsity' of neural spikes naturally leads to fast computation on GPUs is shown to be an illusion. The study tested five distinct sparse computation strategies on an Apple M3 Max GPU and found that none could outperform a standard dense convolution, because modern SIMD hardware cannot efficiently exploit the fine-grained, random sparsity of SNN activity.

To solve this, the author proposes Temporal Aggregated Convolution (TAC). Instead of processing each time step individually, TAC exploits the linearity of convolution to pre-aggregate multiple spike frames (K) before a single computation, reducing work from T calls to T/K. On rate-coded datasets like MNIST, this achieves a dramatic 13.8x speedup while simultaneously improving accuracy by +1.6%. For event-based data with critical motion information, a variant called TAC-TP preserves temporal resolution, achieving 95.1% accuracy with 50% fewer computations. The key insight is data-dependence: collapse time for static data, preserve it for dynamic data.

The speedup is hardware-agnostic, verified with an 11.0x gain on an NVIDIA V100 GPU, proving the method's broad applicability. All operators are open-sourced in the mlx-snn library, providing a practical toolkit for the AI research community to build faster, more accurate neuromorphic models.