Granularity Paradox: Finer Data Hurts Forecast Accuracy, New Study Shows

Hugo Moreira's new study on the 'Granularity Paradox' in time-series forecasting, published on arXiv, challenges the common assumption that more granular data leads to better predictions. The paper systematically benchmarks 10 models—ranging from naïve and statistical to machine learning and deep learning—across six temporal granularities using a 13-year public procurement dataset. Key findings reveal that finer temporal disaggregation (e.g., moving from monthly to daily data) significantly improves in-sample diagnostics but severely degrades out-of-sample accuracy due to recursive error compounding over longer horizons. For instance, the Holt-Winters model at daily granularity achieved a test R-squared of -151 and a TPFE (total percentage forecast error) of 425.85%, indicating catastrophic failure. In contrast, the LSTM (long short-term memory) neural network exhibited a U-shaped error curve, worsening from 19.66% at monthly to 35.94% at bi-weekly before recovering to 4.35% at daily grain. Linear regression remained stable across all granularities (16.3-17.0% TPFE), confirming that the paradox is driven by recursive feedback topology rather than model complexity.

The study provides critical insights for practitioners: standard pointwise metrics like RMSE and MAE systematically mask cumulative error propagation, leading to misleading assessments of model adequacy. Moreira introduces a consensus-dissensus diagnostic that compares directional behavior of pointwise metrics against cumulative TPFE across granularities, enabling identification of models whose standard diagnostics hide systematic error. The results imply that time-series forecasters must carefully select temporal granularity based on forecast horizon and model type, especially when using recursive autoregressive or seasonal models. For professionals building forecasting systems—whether for demand planning, financial markets, or energy loads—this research highlights that simply adding finer data can backfire. The LSTM's unique U-shaped behavior suggests deep learning architectures may partially mitigate the paradox, but only under certain conditions. The paper is a must-read for anyone relying on time-series predictions to make data-driven decisions.