Elastic Spiking Transformer adapts model size at runtime without retraining

Researchers from the University of Trento and affiliated labs have unveiled the Elastic Spiking Transformer, a novel architecture for Spiking Neural Networks (SNNs) that brings runtime adaptability to neuromorphic computing. Traditional SNN models are static—once trained, their parameter count and computational graph are fixed, forcing developers to trade accuracy for feasibility when deploying on hardware like Intel Loihi or SpiNNaker with strict memory budgets. The Elastic Spiking Transformer solves this by embedding nested elasticity into its Feature Extractor, Spiking Self-Attention, and Feed-Forward blocks, inspired by Matryoshka-style representation learning.

Through granularity-aware weight sharing, a single trained model can dynamically slice its network width and number of attention heads at inference time without any retraining. This yields two key advantages: the model can shrink its parameter footprint to match different hardware memory constraints, and reducing active neurons proportionally lowers spike firing rates and synaptic operations—an energy benefit unique to SNNs. Tested on CIFAR10/100, CIFAR10-DVS, and the clinical EHWGesture dataset, the architecture matches or surpasses independently trained baselines across a broad range of complexity-accuracy trade-offs. This enables adaptive, real-time gesture recognition on edge devices, opening the door to more flexible neuromorphic applications in healthcare and beyond.