GLLMs: New grading framework boosts LLMs with zero extra inference cost

The paper proposes Graded Large Language Models (GLLMs), extending graded neural networks to autoregressive transformers. The key idea is to endow the embedding space with a grading—a decomposition into weight spaces indexed by a torus—and propagate a weighted scalar action through attention and training. This algebraic structure is rooted in geometric invariant theory: the benefit of a grading is captured by a Kempf–Ness functional, and optimal grades are found as the coincidence point of two moment maps, solvable offline via convex optimization. Critically, the framework preserves the original transformer architecture after training—every GLLM compiles to a standard transformer with identical computational complexity and inference cost.

The practical impact is twofold. First, for level-stratified data, the paper proves a minimax separation: graded models achieve exponentially lower risk in the number of levels compared to uniform transformers, across an explicit window of sample sizes. This means tasks with hierarchical structure (e.g., code, mathematics, legal documents) see outsized benefits. Second, because the optimal grade direction solves a convex program certified before training, practitioners can precompute the best grades without additional runtime overhead. The ordinary transformer is shown to be a boundary case (a semistable isotropic point) in a larger family, suggesting that current architectures leave significant room for improvement through this mathematically grounded grading approach.