UniTok and UniTok-FM: A Universal Tokenizer for Time Series AI
Treating time series like language unlocks zero-shot forecasting and in-context learning.
Time series data has long resisted the unifying power of large language models due to its continuous, unbounded nature. A new paper from Yunhao Zhang, Ruiying Qi, and colleagues introduces UniTok, a universal tokenizer that discretizes time series into tokens suitable for next-token prediction (NTP). UniTok employs a vector-quantized autoencoder with three key innovations: prefix normalization to stabilize scale variations, a progressive-resolution causal architecture for efficient encoding/decoding, and a structure-preserving reconstruction loss. This transforms arbitrary time series into a discrete vocabulary that an LLM can process natively.
Building on UniTok, the team presents UniTok-FM, a foundation model pretrained via NTP on context windows formed by multiple series with similar patterns—capturing shared dynamics rather than isolated series. Crucially, UniTok-FM uses standard LLM architecture without any time-series-specific modifications. In experiments spanning forecasting, generation, and classification, a single unified model consistently outperforms statistical and supervised baselines, achieves competitive results with task-specific foundation models, and uniquely enables training-free in-context inference across all three tasks. This represents a significant step toward a single, general-purpose time series foundation model.
- UniTok uses a vector-quantized autoencoder with prefix normalization, progressive-resolution causal architecture, and structure-preserving loss to discretize continuous time series.
- UniTok-FM adopts off-the-shelf LLM architecture, pretrained via next-token prediction on multi-series context windows to capture shared temporal dynamics.
- Achieves zero-shot forecasting, few-shot generation, and classification—outperforming statistical baselines and task-specific models without fine-tuning.
Why It Matters
Unifies forecasting, generation, and classification under a single LLM framework, enabling in-context learning for time series.