STARS method boosts SNN accuracy up to 6.7% without original training data

Spiking Neural Networks (SNNs) promise ultra-low-power inference, but their accuracy still trails traditional Artificial Neural Networks (ANNs). Knowledge distillation from ANNs helps, but often requires the original training data — a luxury in production. Researchers from multiple institutions now propose STARS, a data-free distillation method tailored for SNNs.

STARS tackles the core challenge: existing data-free methods use batch normalization (BN) statistics from the teacher, but these only constrain mean and variance — missing the SNN's threshold-crossing dynamics. STARS introduces two new objectives: Relational Consistency Alignment preserves cross-sample relationships between teacher and student, and Tail-Aware Regularization pushes synthetic data to have spike probabilities near the teacher's thresholds. Together they create synthetic batches that are more informative for SNN students.

Tested across CIFAR-10, CIFAR-100, and Tiny-ImageNet with multiple ANN-SNN pairs, STARS consistently outperformed conventional DFKD baselines — hitting gains of up to 4.6% on CIFAR-10 and 6.7% on CIFAR-100. It even surpassed several knowledge distillation methods that had access to real data. The approach is plug-and-play, making it easy to integrate into existing workflows.