New PhD thesis proves AI models hit a hard accuracy ceiling
The scaling hypothesis has guided AI investment for years, but a new thesis proves that transformer architectures have an inherent accuracy limit—a ceiling that data and compute alone cannot surpass.
University of Hong Kong researcher Dongxin Guo has published a sweeping PhD thesis that turns classic impossibility results — from Turing and Arrow to No Free Lunch theorems — into concrete design rules for AI systems. The centerpiece is the Deterministic Horizon, a provable accuracy ceiling determined solely by a transformer's architecture (layer count and embedding width). Guo measures this ceiling across 12 popular transformer architectures and finds it ranges from 19 to 31. Crucially, no amount of additional training data, higher adapter ranks, larger sample sizes, or alternative loss functions can push accuracy past that limit. Fine-tuning on optimal-length traces recovers under four percentage points at best.
The thesis extends the same impossibility-specification methodology to multiple subfields: preference learning under model misspecification shows discontinuous jumps in sample complexity; multi-stage retrieval pipelines require at least as many independent metrics as stages; truthful auctions fail for agents with prompt-dependent valuations; and zero-knowledge verification of neural inference incurs a measured overhead of 110x to 190x per non-linear activation. Together, these form a catalogue of 16 specifications, each pairing a computable boundary, a quantified violation cost, and a constructive design rule. Guo offers this as a generative research programme for trustworthy AI: every fundamental limit is also a design rule.
- Transformer accuracy has a theoretical maximum determined by architecture (layer count and embedding width), ranging from 19 to 31 across common designs.
- Current models are far from this ceiling, so scaling continues to work for now—but the bound sets a long-term limit on performance gains from brute-force expansion.
- The finding challenges the 'more is always better' ethos of scaling, suggesting that future breakthroughs will require architectural innovation, not just larger models.
Why It Matters
A rigorous proof that accuracy is bounded by architecture reshapes the debate on scaling limits and AI investment.