# The Pitfall of Scaling Up: Uncovering and Mitigating Popularity Bias Amplification in Scaling Transformer-based Recommenders Source: [The Pitfall of Scaling Up: Uncovering and Mitigating Popularity Bias Amplification in Scaling Transformer-based Recommenders]() Collection index: [llms.txt]() Data updated: 2026-06-24T05:00:13.022Z Authors: Weiqin Yang, Yue Pan, Chongming Gao, Sheng Zhou, Xiang Wang, Can Wang, Jiawei Chen Published: 2026-06-20T07:13:49.000Z ## Abstract We identify a critical pitfall in scaling transformer-based sequential recommenders: while increasing model size improves recommendation accuracy, it simultaneously amplifies popularity bias. This bias drives systems to over-recommend popular items at the expense of niche ones, which not only undermines fairness but also degrades the broader ecosystem by reinforcing the Matthew effect and filter bubbles. Consequently, this bias amplification emerges as a fundamental obstacle to sustainable model scaling. Through comprehensive theoretical and empirical analyses, we uncover the root cause of this amplification. Our findings reveal that as model depth increases, the two core components of the transformer architecture, i.e., attention aggregation and feed-forward projections, synergistically induce severe spectral collapse in model predictions, which directly translates to the amplification of popularity bias. To address this challenge, we propose SPRINT (Scalable Popularity Regularization IN Transformers), which mitigates spectral collapse during scaling by constraining (i) the maximum column-sums of the attention score matrices and (ii) the spectral norms of the feed-forward parameters. Extensive experiments demonstrate that SPRINT significantly improves both accuracy and long-tail fairness. Crucially, it yields more favorable scaling behaviors when expanding model sizes from 0.05M to 0.34B parameters. The code is available at https://github.com/Tiny-Snow/GenRec. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Sequential Recommendation on Beauty (NDCG@10, HitRatio@10, Fair-0.8)](): Sequential Recommendation; Beauty; NDCG@10: 4.13. - [Sequential Recommendation on Toy (NDCG@10, HitRatio@10, Fair-0.8)](): Sequential Recommendation; TOY; NDCG@10: 4.85. - [Sequential Recommendation on Electronic](): Sequential Recommendation; Electronic; NDCG@10: 2.44. - [Sequential Recommendation on Book](): Sequential Recommendation; Book; NDCG@10: 3.89. - [Sequential Recommendation on ML-1M (NDCG@5, HitRatio@5, Fair-0.8)](): Sequential Recommendation; ML 1M; NDCG@5: 9.25. - [Sequential Recommendation on Beauty (NDCG@5, HitRatio@5, Fair-0.8)](): Sequential Recommendation; Beauty; NDCG@5: 3.62. - [Sequential Recommendation on Toy (NDCG@5, HitRatio@5, Fair-0.8)](): Sequential Recommendation; TOY; NDCG@5: 4.38. - [Sequential Recommendation on Clothing (NDCG@5, HitRatio@5, Fair-0.8)](): Sequential Recommendation; Clothing; NDCG@5: 0.0036. - [Sequential Recommendation on Book (K=5 Metrics)](): Sequential Recommendation; Book; NDCG@5: 3.47. - [Generative Recommendation on Beauty (NDCG@5, HR@5, Fair)](): Generative Recommendation; Beauty; NDCG@5: 3.43. - [Generative Recommendation on Toy](): Generative Recommendation; TOY; NDCG@5: 0.0341. - [Sequential Recommendation on Clothing](): Sequential Recommendation; Clothing; HR@10: 0.75. ## Original sources and implementation [Original paper]() [PDF]()