EVOTS uses evolution to design better Transformers for time series forecasting

A new paper from researchers AbdElRahman ElSaid and Damir Pulatov proposes EVOTS (Evolutionary Transformer Search), a framework that applies evolutionary algorithms to automatically design Transformer-like architectures for time series forecasting. Unlike most current approaches that rely on fixed, hand-crafted Transformer architectures, EVOTS encodes models using a modular genome representation that allows flexible composition of attention, feed-forward, and projection components. A repair mechanism enforces structural validity during evolution, enabling effective exploration of a diverse architecture space without needing predefined design rules.

The method was evaluated on four benchmark datasets from the ETT family (ETTh1, ETTh2, ETTm1, ETTm2) across multiple forecasting settings: univariate-to-univariate, multivariate-to-univariate, and multivariate-to-multivariate prediction, with forecasting horizons of 96, 192, 336, and 720 time steps. In the key multivariate-to-multivariate setting, evolved architectures achieved competitive and often improved mean squared error compared to a strong Transformer baseline. Additional analysis examines performance differences across settings and reports wall-clock training time as an indication of computational cost. The results demonstrate that evolutionary search can practically discover high-performing, flexible architectures for time series forecasting within reasonable runtime constraints.