New 'Thermostat' for AI Training Makes Models Suddenly Get It Faster

There's a strange moment in AI training called "grokking." A model can look hopeless for a long time, failing on a task it has technically seen thousands of times, and then — seemingly out of nowhere — it suddenly gets it. It stops memorizing and starts following the real underlying rule. Nobody has had a reliable way to trigger that moment, so companies mostly just wait, burning expensive computing time and electricity while a model spins its wheels.

A new paper from researchers Luan Ozelim and Hector Zenil describes a smarter approach: measure how complex the model's internal rulebook is at any given moment, then step in only when that measurement says the model is ready to make the leap. Think of it like a thermostat that fires the furnace at exactly the right temperature instead of running it constantly. They call their tool a "differentiable complexity controller" — jargon, but the idea is simple: a dial you can turn, guided by a signal, rather than guesswork.

In their experiments, this timing-based nudge sped up grokking and, notably, rescued training runs that were otherwise heading for failure. Their complexity signal did that with 27% less intervention than simply watching the standard training error. The trick also transferred to a different kind of problem and to a transformer, the architecture behind ChatGPT-style models. Importantly, the researchers found the value is in the timing — when to push and when to stop — not in fancy attribution of which parts of the model deserve credit.

What's the catch? This is early, small-scale research, not a product. The experiments involved controlled tasks, not giant frontier models, and one method they tried (a sustained loss in the weight space) simply didn't work. They also can't yet explain the underlying physics cleanly. Still, the direction matters: if training gets more efficient, AI gets cheaper to build, which eventually reaches your wallet — and the power grid.