S3T-Former: A Purely Spike-Driven State-Space Topology Transformer for Skeleton Action Recognition

A research team has introduced S3T-Former, a novel neuromorphic architecture designed to bring efficient AI to edge devices for skeleton-based action recognition. This task, crucial for applications like surveillance, human-computer interaction, and AR/VR, has traditionally relied on power-hungry Artificial Neural Networks (ANNs). While Spiking Neural Networks (SNNs) offer an energy-efficient alternative, existing models often compromise their inherent sparsity or suffer from short-term memory issues. S3T-Former directly addresses these limitations by being the first purely spike-driven Transformer built specifically for this domain.

At its core, S3T-Former employs a Multi-Stream Anatomical Spiking Embedding (M-ASE) module. This acts as a generalized kinematic differential operator, elegantly transforming multimodal skeleton data—like joint positions and velocities—into highly sparse, heterogeneous event streams. To manage this sparsity effectively, the model features Lateral Spiking Topology Routing (LSTR) for on-demand conditional spike propagation. Most critically, it integrates a Spiking State-Space (S3) Engine, a novel component that systematically captures long-range temporal dynamics without resorting to non-sparse spectral transformations that break the spike-driven paradigm.

Extensive testing on multiple large-scale datasets demonstrates that S3T-Former achieves accuracy competitive with traditional ANNs. Its true breakthrough, however, is its theoretical energy efficiency. By maintaining topological and temporal sparsity throughout its processing, the architecture minimizes the dense matrix operations that drain power. This establishes a new state-of-the-art for deploying advanced action recognition on battery-powered devices, from smart glasses to drones, without sacrificing performance.