Scaling transformers amplifies popularity bias – new SPRINT method fixes it

A team of researchers discovered that scaling up transformer-based recommender systems creates a hidden trade-off: larger models become more accurate but increasingly recommend popular items at the expense of niche ones. The problem stems from “spectral collapse,” where deep transformers concentrate predictions into a narrow subspace, directly amplifying popularity bias. Through theoretical and empirical analysis, they showed that both attention aggregation and feed-forward projections contribute to this effect as model depth increases.

To counter this, the team introduces SPRINT (Scalable Popularity Regularization IN Transformers). SPRINT constrains the maximum column-sums of attention score matrices and the spectral norms of feed-forward parameters, preventing spectral collapse without sacrificing accuracy. Extensive experiments across model sizes from 0.05M to 0.34B parameters demonstrate that SPRINT significantly improves both accuracy and long-tail fairness. The paper has been accepted at KDD 2026, and the code is publicly available.