Meta's 60.2T token leaderboard highlights missing ROI in agentic AI workflows
Meta tracked 85,000 employees' token usage, but agents aren't getting cheaper with repetition.
Meta's internal experiment with a token consumption leaderboard exposed a deeper industry issue: we're measuring AI adoption by raw compute usage, not efficiency. The company tracked how many tokens each of its 85,000 employees burned, with top users earning 'Token Legend' badges after consuming 60.2 trillion tokens in 30 days. The leaderboard was eventually taken down because employees started gaming the rankings. Jensen Huang has said he'd be 'deeply concerned' if engineers weren't spending heavily on AI compute, but the post argues this mentality conflates consumption with productivity.
Bill Gates once quipped that measuring software progress by lines of code is like measuring airplane construction by weight. The same fallacy applies today: token volume says nothing about value delivered. The post introduces ROTI (Return on Token Investment), arguing that a mature agentic workflow should use fewer tokens over repeated runs. If an agent actually learns a task, the 10th run should be faster and cheaper than the first. Most agents don't do this—token spend remains flat regardless of repetition. Without tracking that signal, organizations are just renting compute on repeat rather than building leverage. The question remains: what are you using to decide if an agent is pulling its weight?
- Meta tracked 85,000 employees consuming 60.2 trillion tokens in 30 days via an internal leaderboard that was later removed due to gaming.
- Current AI adoption metrics treat token volume as a proxy for output, ignoring efficiency gains from repeated workflows.
- The proposed ROTI metric (Return on Token Investment) would reveal whether agents actually learn—token spend should drop with repetition.
Why It Matters
Without token ROI metrics, enterprises risk paying for repeated compute without efficiency gains from true learning.