ADSEQ framework boosts SNN training with delay-aware autograd queues
New framework enables differentiable SNNs with efficient spike event queues across CPUs, GPUs, TPUs, and LPUs.
ADSEQ (delay-aware autograd-compatible spike event queues) tackles a fundamental bottleneck in training Spiking Neural Networks (SNNs): efficiently handling sparse spike events and time delays during gradient computation. Traditional SNN simulators use dense, memory-heavy data structures or ignore delays altogether, limiting scalability and compatibility with modern deep learning frameworks. ADSEQ derives gradient computation through spike event queues that explicitly model delays, and implements them in memory-efficient, autodifferentiable data structures. This allows SNNs to be trained using standard backpropagation (autograd) without sacrificing the temporal sparsity that makes them attractive for energy-efficient neuromorphic computing.
The framework was benchmarked across CPU, GPU, TPU, and LPU (Groq's Language Processing Units), revealing that queue design significantly impacts performance. CPUs perform best with traditional tree-based or FIFO implementations; GPUs excel with ring buffers for smaller simulations but shift to sparser data structures under memory pressure; TPUs favor sorting-based intrinsics. The paper also introduces selective spike dropping as a simple knob for trading accuracy for speed—a technique that could be further enhanced by future autograd frameworks that adapt diverging primal/tangent data structures. These findings provide practical guidance for implementing SNN training on modern AI accelerators.
- ADSEQ enables gradient computation through spike event queues that explicitly include delays, improving temporal fidelity in SNN training.
- Benchmarked on CPU, GPU, TPU, and LPU: queue design is platform-specific (e.g., ring buffers for GPU, sorting intrinsics for TPU).
- Selective spike dropping provides a straightforward performance-accuracy trade-off, with potential for optimization in future autograd frameworks.
Why It Matters
ADSEQ unlocks efficient, differentiable SNN training across diverse hardware—critical for scaling neuromorphic AI to real-world, energy-constrained applications.