AI Training Just Got Simpler: One Setting Works at Any Depth

Most AI today is trained as one giant connected machine: information flows forward, a single error signal flows all the way back, and every part adjusts at once. A rival approach called "local learning" trains each layer separately, like an assembly line where every station checks its own work instead of waiting for the finished product. That makes training easier to run in parallel — but it has a nasty flaw. The deeper the network gets, the worse the accuracy becomes, and the settings that worked at one size stop working at another. So almost nobody uses it.

This paper applies a known trick — roughly, forcing each layer's updates into a cleaner, more balanced shape — to those per-layer updates. The result: one step-size setting (think of it as how big a stride the AI takes while learning) worked best across every width and depth tested. The old standard method needed re-tuning at every depth and still collapsed at depth 48, scoring about 19 percent accuracy when it carried its shallow-network setting forward. The new method hit 42.7 percent at the same unchanged setting.

The author also reframes the step size as a "drift contract": the AI takes smaller steps when its input is large, so no single layer can change too violently in one go. That makes the setting interpretable rather than magical, and gives a per-layer safety bound that ordinary optimizers don't offer.

The honest catch: these are tiny tests on CIFAR-10, a classic small image dataset, using simple networks — not the giant models behind ChatGPT. And the author reports a negative result: when a common stabilizer called RMSNorm is added, the advantage shifts to global training rather than local. So the benefit appears exactly where that stabilizer is missing. Promising, but far from proven at real-world scale.