Memory-Augmented Autoencoder Hits 98.4% Accuracy in Unsupervised Activity Recognition

Human activity recognition (HAR) from IMU sensors is critical for healthcare monitoring and rehabilitation, but traditional deep learning methods struggle with reliance on labeled data, multi-sensor fusion complexity, and capturing spatiotemporal dependencies in noisy, real-world scenarios. A new paper from Saeid Arabzadeh and colleagues proposes a fully unsupervised framework that sidesteps these issues using a memory-augmented autoencoder. The architecture works in two stages: first, a stacked autoencoder extracts hierarchical static features from short temporal windows of multi-sensor IMU data, fusing information both within and across sensors. Then, a sequence-to-sequence LSTM autoencoder refines these features temporally, learning historical motion patterns without any labels. This design allows the model to generate discriminative activity representations suitable for real-time applications.

The method was evaluated on the DaLiAc and PAMAP2 datasets using realistic inter-class window segmentation, which intentionally introduces activity transitions that reduce accuracy by ~7% but better reflect real-world conditions. Despite this, the framework achieved 96.6% and 98.4% accuracy respectively, outperforming both supervised baselines and existing unsupervised approaches. Notably, it improved feature separability by up to 9% even when constrained to short temporal windows. The researchers analyzed key hyperparameters to maximize separability, demonstrating that the memory-augmented architecture is robust to noisy data and overlapping activities. This work paves the way for label-free HAR systems that can be deployed on wearable devices for continuous health monitoring and rehabilitation without the overhead of manual annotation.