Decentralized Attention Fails Centralized Signals: Rethinking Transformers for Medical Time Series

A research team led by Guoqi Yu has published a breakthrough paper accepted as an Oral presentation at ICLR 2026, introducing CoTAR (Core Token Aggregation-Redistribution) - a novel AI architecture specifically designed for medical time series analysis. The work addresses a fundamental limitation of Transformer models when applied to medical signals like EEG and ECG: while Transformers excel at capturing temporal dependencies, their decentralized attention mechanism struggles with the centralized, synchronized nature of multi-channel medical data where global waveform patterns matter.

The technical innovation replaces standard Transformer attention with a centralized MLP-based module that introduces a global core token as a proxy for inter-token interactions. This architectural shift better aligns with the inherent structure of medical signals while dramatically reducing computational complexity from quadratic to linear. Experimental results across five medical benchmarks show CoTAR achieving up to 12.13% improvement on the APAVA dataset while using only 33% of the memory and 20% of the inference time compared to previous state-of-the-art approaches.

This research represents a significant departure from simply applying general-purpose Transformer architectures to medical domains. By designing AI models that respect the specific structural properties of medical data, the team has demonstrated that domain-aware architectural choices can yield both performance gains and efficiency improvements. The work has immediate implications for real-time medical monitoring systems and could enable more accurate, resource-efficient diagnostic tools for brain and heart diseases. All code and training scripts have been made publicly available, facilitating further research and potential clinical applications.