DoTime's synthetic benchmark shows interventional training beats observational models

DoTime, a new open-source benchmark generator from researchers including Dennis Thumm, tackles a critical gap in causal inference for time series: the scarcity of interventional and counterfactual data. Most existing benchmarks are observational, small, or domain-specific, which limits progress in fields like healthcare, policy evaluation, and climate science. DoTime generates multivariate temporal structural causal models (TSCMs) with interventions, packaged as the dotime PyPI library alongside four frozen evaluation suites. It adds capabilities missing from prior generators, including continuous-time intervention windows, counterfactual sampling with a positivity guard, regime-switching SCMs, non-stationary dynamics, and deterministic ramp/sinusoidal intervention profiles that embed trends and structural breaks inside the evaluation window.

The released suites cover 100,000 trajectories and eight named identification structures, each with exact ground truth: paired interventional trajectories from the same SCM and shared-noise counterfactuals in the continuous-time suite. DoTime also ships reference baseline implementations with an evaluation harness, enabling reproducible comparisons. The paper's central claim is falsifiable and tested: interventional training delivers a measurable direction-accuracy advantage over an observational model of identical capacity. Across three training seeds per arm, the interventional prior-fitted network (PFN) showed a positive gap in every structure, trajectory length, and seed tested under structure-matched evaluation on held-out episodes. This makes DoTime a promising prior source for causal foundation models and a robust testbed for validating causal AI systems before deployment in high-stakes domains.