Beyond Coefficients: Forecast-Necessity Testing for Interpretable Causal Discovery in Nonlinear Time-Series Models
New framework challenges how we trust AI's causal claims, using systematic ablation on real-world data.
A team of researchers has published a significant paper titled 'Beyond Coefficients: Forecast-Necessity Testing for Interpretable Causal Discovery in Nonlinear Time-Series Models' on arXiv. The work, led by Valentina Kuskova, Dmitry Zaytsev, and Michael Coppedge, tackles a critical flaw in how we interpret complex AI models like neural networks used for causal discovery. Currently, outputs from models like regularized neural autoregressive models are often misinterpreted, with causal scores treated like traditional regression coefficients, leading to potentially false claims of statistical significance.
The researchers propose a paradigm shift: evaluate causal relevance through 'forecast necessity' rather than coefficient magnitude. Their practical framework involves systematically ablating (removing) candidate causal edges in a model and comparing forecast accuracy. If removing a link doesn't harm prediction, it's not a necessary causal relationship. They demonstrated this using a Neural Additive Vector Autoregression model on a high-stakes, real-world dataset: a multivariate time series of democracy indicators across 139 countries.
The results were revealing. The study showed that candidate relationships with similar causal scores could differ dramatically in their actual predictive necessity. This discrepancy arises from factors like redundancy (other variables providing the same information), temporal persistence, and effects specific to certain political regimes. The new testing procedure provides a more reliable method for causal reasoning in applied AI systems, offering concrete guidance for practitioners in fields like economics, political science, and public policy who rely on nonlinear time-series models.
- Proposes 'Forecast-Necessity Testing'—a new evaluation framework based on systematic edge ablation and forecast comparison to test if a causal link is necessary.
- Demonstrates the method on a Neural Additive Vector Autoregression model using real-world panel data of democracy indicators from 139 countries.
- Finds that relationships with similar model scores can have vastly different predictive importance due to redundancy, temporal persistence, and regime-specific effects.
Why It Matters
Provides a more reliable method for interpreting AI-driven causal claims in high-stakes domains like economics and policy, moving beyond misleading coefficient-based analysis.