AI Watchdogs Can Now Count Training Work by Watching Power Use

As AI models get more powerful, countries are talking about agreements to keep them safe. But a big problem: how do you define a "frontier" model? Rules like the EU AI Act and California's SB 53 use the number of calculations, or FLOPs, done during training. The catch is that those numbers currently come straight from AI companies themselves. That works if everyone is honest, but international rivals won't just trust each other's word.

This new research tests a way to check those numbers from the outside. An independent "verifier" gets access to a computer system but not the AI's secret code or data. Instead, they watch side channels: how much electricity the graphics cards use, how busy the memory is, and how much data flows between chips. By calibrating those readings against known training examples, they estimated total workload within about 10% of the true figure for typical training runs.

Of course, a company that wants to hide its AI's true power could fight back. The researcher also played the role of a cheater, designing training tricks that made the estimates worse. Under the strongest evasion strategy, the median error jumped to about 25%, with one configuration under-reporting by 41%. That means the method is promising but not foolproof.

Still, this is an important step toward real verification in AI policy. It shows that independent checks are physically possible without forcing companies to reveal trade secrets. Future treaties may not need to rely on trust alone — they could use power meters and memory monitors as honest referees.