New Math Explains Why AI Training Eventually Settles Down

Two researchers, Stéphane Galatolo and Stéphane Chrétien, published a new math paper on arXiv about how AI learns. The algorithm they studied, called stochastic gradient descent, is the workhorse behind nearly every modern AI system. Training an AI with it is a bit like a hiker lost in fog, trying to find the lowest point in a valley by taking small, slightly random steps downhill. Their question was simple: how long until the hiker gets there?

The answer is a familiar pattern. The time it takes to reach a good solution follows what statisticians call an exponential distribution — short waits are common, very long waits are rare but possible. Think of waiting for a bus: most arrive soon, but occasionally you stand there forever. The researchers show the average waiting time is tied to how often the algorithm naturally lingers near that good spot. Their proof covers the two most common types of randomness used in AI training.

Why should you care? This is pure theory — no product, no app, no price change. But it matters indirectly. Training a large AI model can cost millions of dollars in electricity and specialized chips. Knowing that training times follow a predictable curve, rather than being random luck, could help companies schedule their work, budget their computing bills and decide when to stop pouring money into a run that isn't improving.

The catch: the math depends on assumptions. The random noise in the algorithm has to behave in certain tidy ways, and real-world AI training rarely matches textbook conditions exactly. So this explains the ideal case, not every messy training run in practice. It's also a preprint, meaning it hasn't yet been checked and approved by other scientists. Useful scaffolding, but not the finished building.