Marc Schmitt's Algometrics proves forecasting alters the future it predicts

Marc Schmitt's new paper, "Algometrics: Forecasting Under Algorithmic Feedback," tackles a fundamental blind spot in algorithmic markets: when a predictive model's outputs drive trades, allocations, or risk controls, they alter the very data the model will later be evaluated on. Schmitt formalizes this feedback loop, distinguishing between historical risk (measured under passive forecasting) and deployment risk (measured when forecasts actually trigger actions).

Schmitt proves three critical results. First, deployment risk cannot be inferred from passive historical data alone—even in a simple one-step linear feedback model, infinitely many algorithm-mediated environments produce the same historical data but carry drastically different deployment risks. Second, model rankings can invert under crowding: a forecaster with lower passive error may end up with higher deployment error once similar algorithms are adopted by the market. Third, using randomized or instrumented actions can identify short-horizon linear feedback, and Schmitt derives a finite-sample bound for estimating deployment risk. These findings suggest that time‑series benchmarks in algorithmic markets should report feedback sensitivity alongside traditional predictive accuracy.