MSGL-Transformer: A Multi-Scale Global-Local Transformer for Rodent Social Behavior Recognition
New transformer model achieves 10.7% accuracy boost over previous state-of-the-art for neuroscience research.
Researchers Muhammad Imran Sharif and Doina Caragea have introduced MSGL-Transformer, a novel AI architecture designed to automate the recognition of rodent social behaviors from pose-based video data. The model addresses the critical bottleneck in neuroscience research: manual behavior scoring is notoriously slow and subjective. MSGL-Transformer processes temporal sequences of animal poses using a lightweight transformer encoder equipped with a multi-scale attention mechanism. Its key innovation is the parallel integration of short-range, medium-range, and global attention branches, allowing it to explicitly capture motion dynamics and social interactions happening at different timescales, from quick sniffs to prolonged grooming.
The architecture is further enhanced by a Behavior-Aware Modulation (BAM) block, inspired by SE-Networks, which acts as a feature filter. Before the attention mechanism processes the data, the BAM block modulates the temporal embeddings to emphasize the features most relevant to specific behaviors, improving the model's focus and efficiency. Evaluated on two standard benchmarks, RatSI and CalMS21, the model demonstrated strong performance and generalization. It achieved a mean accuracy of 75.4% on RatSI and a standout 87.1% accuracy on CalMS21, representing a significant +10.7% improvement over the previous best model, HSTWFormer.
This performance gain means MSGL-Transformer reliably outperforms several established deep learning baselines, including Temporal Convolutional Networks (TCN), various LSTMs, and spatial-temporal graph convolutional networks like ST-GCN. The same core architecture successfully adapted to both datasets by only changing the input pose dimensionality and number of behavior classes, proving its flexibility. By providing a fast, objective, and highly accurate alternative to human scoring, this tool can drastically accelerate the pace of behavioral neuroscience and psychopharmacology research.
- Achieves 87.1% accuracy on CalMS21 dataset, a +10.7% improvement over prior state-of-the-art model HSTWFormer.
- Uses a novel multi-scale attention mechanism with parallel branches to capture behavior dynamics across short, medium, and long temporal ranges.
- Incorporates a Behavior-Aware Modulation (BAM) block to filter and emphasize the most relevant features for accurate classification.
Why It Matters
Automates a tedious, error-prone task in labs, enabling faster and more scalable analysis of animal behavior for drug discovery and neuroscience.