TSFMAudit detects hidden data contamination in time series AI models

Data contamination—where evaluation data leaks into pretraining—is a growing problem for AI models. For text-based LLMs, exact-match checks offer some protection, but time series signals are continuous and heterogeneous, making contamination auditing far harder. A new paper from researchers at Zhejiang University, Salesforce AI, and others proposes TSFMAudit, the first formal framework to detect such leakage in time series foundation models (TSFMs). The method leverages a simple but powerful insight: when a model has already seen a dataset during pretraining, fine-tuning on that same data shows unusually efficient adaptation—loss drops faster and the model’s weights change less. TSFMAudit measures these signals via probe adaptation dynamics, flagging potentially contaminated evaluation sets.

The team evaluated TSFMAudit across 6 major TSFMs (including Lag-Llama, PatchTST, and TimesNet) and 187 benchmark datasets, using documented training sources as ground-truth labels for contamination. Compared to 10 baseline methods from the LLM contamination literature, TSFMAudit achieved the highest detection accuracy, demonstrating that time series contamination leaves a distinct adaptive signature. The authors also release their code and benchmarks. This work is critical as TSFMs become more widely deployed in finance, energy, and weather forecasting—domains where inflated performance claims could lead to costly real-world decisions. By giving auditors a reliable detection tool, TSFMAudit helps ensure that reported accuracy reflects genuine generalization, not data leakage.