Learning with the Nash-Sutcliffe loss
New 77-page paper proves a widely used forecast metric is strictly consistent for a new multi-dimensional functional.
Researchers Hristos Tyralis and Georgia Papacharalampous have published a foundational 77-page paper titled 'Learning with the Nash-Sutcliffe loss' on arXiv. The work provides a rigorous decision-theoretic justification for the widely used Nash-Sutcliffe Efficiency (NSE), a positively oriented relative measure for evaluating forecasts across multiple time series. The authors address a key gap by examining its counterpart, the Nash-Sutcliffe loss (L_NS = 1 - NSE), proving it is strictly consistent for an elicitable and identifiable multi-dimensional functional they term the 'Nash-Sutcliffe functional.' This functional is a data-weighted component-wise mean, and the proof formally validates the common empirical practice of maximizing the average NSE across series.
Technically, the paper shows that maximizing average NSE implicitly assumes all series originate from a single non-stationary stochastic process. To extend the framework, the authors introduce 'Nash-Sutcliffe linear regression,' a multi-dimensional model estimated by minimizing the average L_NS, which reduces to a data-weighted least squares problem. This reorientation allows the framework to be applied more naturally to forecasting multiple stationary dependent time series with differing stochastic properties. The results establish a stronger theoretical foundation for NSE-based model estimation and forecast evaluation in large datasets, while providing new clarity on the comparative advantages of global machine learning models over local ones for such tasks.
- Proves Nash-Sutcliffe loss (L_NS) is strictly consistent for a new 'Nash-Sutcliffe functional,' a data-weighted component-wise mean.
- Introduces 'Nash-Sutcliffe linear regression,' a multi-dimensional model estimated via data-weighted least squares derived from minimizing L_NS.
- Provides formal decision-theoretic foundation for using average NSE, validating common practice and clarifying benefits of global ML models for large datasets.
Why It Matters
Provides the missing theoretical rigor for a cornerstone forecasting metric, guiding better model selection and evaluation in fields like hydrology and climate science.