OmniDecVAEs unify 30-modal wearable AI with 4.1M-parameter model

Wearable devices generate streams of heterogeneous sensor data, but existing AI models struggle to handle many modalities simultaneously while remaining interpretable, efficient, and generative. A team led by Ioannis Ziogas and colleagues from Khalifa University, the University of Toronto, and partner institutions introduces Omni-modal Variational Decomposition Autoencoders (OmniDecVAEs), a framework that learns disentangled, interpretable representations across arbitrarily many sensor channels. Unlike prior approaches, OmniDecVAEs acts as a full-stack wearable processor: it handles task-specific classification (like activity and identity recognition), representation learning for downstream tasks, cross-modal fusion, and generative modeling—all within one unified and scalable architecture.

The model extends DecVAEs by introducing modality-conditioned time-frequency latent subspaces, learned through a multi-view self-supervised decomposition loss and a shared asymmetric autoencoder. In tests on a challenging omni-modal human activity recognition (HAR) benchmark with up to 30 modalities, OmniDecVAEs outperformed transformer-based and VAE-based methods, boosting activity recognition accuracy by 1.01% and identity recognition by 6.75%. It also demonstrates strong generative capabilities: synthetic omni-modal time-frequency data achieves a 76.84% improvement in mean absolute error and a 13.85% improvement in maximum mean discrepancy, indicating near-realistic data synthesis. Crucially, the model remains lightweight with only 4.1M parameters and real-time latency, making it suitable for intelligent edge wearables and clinical healthcare applications that demand privacy-preserving, on-device processing with a single unified model.