New Math Could Make Rare Financial Crises Easier to Predict

Most forecasts are easy when something happens often. But predicting rare events — like a sudden market crash or a machine failure — is tricky because they almost never show up in the data. A new paper by researcher Jaskaran Singh tackles this problem head-on: when you have a huge pile of mostly ordinary examples and only a tiny smattering of rare 'positive' ones, how do you choose sample the data as accurately as possible?

Singh's solution is a mathematical formula for splitting your sample between rare and common cases. The standard approach, called proportional allocation, copies whatever share exists in the real world. But Singh shows that's not the best move. Instead, you should treat both groups roughly equally — even if the rare events make up only 1% of everything. That counterintuitive equal split cancels out the imbalance and actually reduces forecasting error.

To prove it works, he tested the method on real data: 350 U.S. stocks from 2004 to 2011, looking for moments when prices suddenly turned explosive (a classic sign of a bubble or crash risk). Those explosive moments make up under 1% of all daily data. Using his method, predictions for a 10-day horizon lined up perfectly with the theory, achieving a perfect statistical order across all five test designs. In plain terms, the method correctly told which sampling strategies would do best.

The catch: his formulas didn't predict how results would change across different time horizons — only within a fixed forecasting window. That means the approach is still promising for short-term warnings, not yet a crystal ball for every possible crisis. Still, for people who build early-warning systems — in finance, cybersecurity, or healthcare — this offers a way to squeeze more insight from tiny samples of rare disasters, without waiting for the next crash to feed the machine.