Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
194 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

How Do Recommendation Models Amplify Popularity Bias? An Analysis from the Spectral Perspective (2404.12008v6)

Published 18 Apr 2024 in cs.IR and cs.AI

Abstract: Recommendation Systems (RS) are often plagued by popularity bias. When training a recommendation model on a typically long-tailed dataset, the model tends to not only inherit this bias but often exacerbate it, resulting in over-representation of popular items in the recommendation lists. This study conducts comprehensive empirical and theoretical analyses to expose the root causes of this phenomenon, yielding two core insights: 1) Item popularity is memorized in the principal spectrum of the score matrix predicted by the recommendation model; 2) The dimension collapse phenomenon amplifies the relative prominence of the principal spectrum, thereby intensifying the popularity bias. Building on these insights, we propose a novel debiasing strategy that leverages a spectral norm regularizer to penalize the magnitude of the principal singular value. We have developed an efficient algorithm to expedite the calculation of the spectral norm by exploiting the spectral property of the score matrix. Extensive experiments across seven real-world datasets and three testing paradigms have been conducted to validate the superiority of the proposed method.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (71)
  1. {{\{{TensorFlow}}\}}: a system for {{\{{Large-Scale}}\}} machine learning. In 12th USENIX symposium on operating systems design and implementation (OSDI 16), pages 265–283, 2016.
  2. Principal component analysis. Wiley interdisciplinary reviews: computational statistics, 2(4):433–459, 2010.
  3. Controlling popularity bias in learning-to-rank recommendation. In Proceedings of the eleventh ACM conference on recommender systems, pages 42–46, 2017.
  4. The impact of popularity bias on fairness and calibration in recommendation. arXiv preprint arXiv:1910.05755, 2019.
  5. The connection between popularity bias, calibration, and fairness in recommendation. In Proceedings of the 14th ACM Conference on Recommender Systems, pages 726–731, 2020.
  6. Implicit regularization in deep matrix factorization. Advances in Neural Information Processing Systems, 32, 2019.
  7. On measuring popularity bias in collaborative filtering data. In Proceedings of the Workshops of the EDBT/ICDT 2020 Joint Conference, Copenhagen, Denmark, March 30, 2020, volume 2578, 2020.
  8. Towards mitigating dimensional collapse of representations in collaborative filtering. arXiv preprint arXiv:2312.17468, 2023a.
  9. How graph convolutions amplify popularity bias for recommendation? arXiv preprint arXiv:2305.14886, 2023b.
  10. How graph convolutions amplify popularity bias for recommendation? Frontiers of Computer Science, 18(5):185603, 2024.
  11. Bias and debias in recommender system: A survey and future directions. ACM Transactions on Information Systems, 41(3):1–39, 2023c.
  12. Adap-τ𝜏\tauitalic_τ: Adaptively modulating embedding magnitude for recommendation. In Proceedings of the ACM Web Conference 2023, pages 1085–1096, 2023d.
  13. Co-training disentangled domain adaptation network for leveraging popularity bias in recommenders. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 60–69, 2022.
  14. Gradient descent for deep matrix factorization: Dynamics and implicit bias towards low rank. Applied and Computational Harmonic Analysis, page 101595, 2023.
  15. Power-law distributions in empirical data. SIAM review, 51(4):661–703, 2009.
  16. Visual analysis of recommendation performance. In Proceedings of the Eleventh ACM Conference on Recommender Systems, pages 362–363, 2017.
  17. Deep neural networks for youtube recommendations. In Proceedings of the 10th ACM conference on recommender systems, pages 191–198, 2016.
  18. Debiased recommendation with neural stratification. AI Open, 3:213–217, 2022.
  19. News session-based recommendations using deep neural networks. In Proceedings of the 3rd workshop on deep learning for recommender systems, pages 15–23, 2018.
  20. Addressing unmeasured confounder for recommendation with sensitivity analysis. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 305–315, 2022.
  21. Alleviating matthew effect of offline reinforcement learning in interactive recommendation. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, page 238–248, 2023a.
  22. Cirs: Bursting filter bubbles by counterfactual interactive recommender system. ACM Transactions on Information Systems, 42(1):1–27, 2023b.
  23. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the thirteenth international conference on artificial intelligence and statistics, pages 249–256. JMLR Workshop and Conference Proceedings, 2010.
  24. Offline evaluation to make decisions about playlistrecommendation algorithms. In Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining, pages 420–428, 2019.
  25. Neural collaborative filtering. In Proceedings of the 26th international conference on world wide web, pages 173–182, 2017.
  26. Lightgcn: Simplifying and powering graph convolution network for recommendation. In Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval, pages 639–648, 2020.
  27. What recommenders recommend: an analysis of recommendation biases and possible countermeasures. User Modeling and User-Adapted Interaction, 25:427–491, 2015.
  28. Test time embedding normalization for popularity bias mitigation. arXiv preprint arXiv:2308.11288, 2023.
  29. DP Kingma. Adam: a method for stochastic optimization. In Int Conf Learn Represent, 2014.
  30. On sampled metrics for item recommendation. In Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, pages 1748–1757, 2020.
  31. Balancing unobserved confounding with a few unbiased ratings in debiased recommendations. In Proceedings of the ACM Web Conference 2023, pages 1305–1313, 2023.
  32. Tackling long-tailed distribution issue in graph neural networks via normalization. IEEE Transactions on Knowledge and Data Engineering, 2023.
  33. Improving graph collaborative filtering with neighborhood-enriched contrastive learning. In Proceedings of the ACM Web Conference 2022, pages 2320–2329, 2022.
  34. Seppo Linnainmaa. Taylor expansion of the accumulated rounding error. BIT Numerical Mathematics, 16(2):146–160, 1976.
  35. Popularity debiasing from exposure to interaction in collaborative filtering. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, page 1801–1805, 2023.
  36. Feedback loop and bias amplification in recommender systems. In Proceedings of the 29th ACM international conference on information & knowledge management, pages 2145–2148, 2020.
  37. Collaborative prediction and ranking with non-random missing data. In Proceedings of the third ACM conference on Recommender systems, pages 5–12, 2009.
  38. Probabilistic matrix factorization. Advances in neural information processing systems, 20, 2007.
  39. Curse of "low" dimensionality in recommender systems. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, page 537–547, 2023.
  40. Judea Pearl. Causality. Cambridge university press, 2009.
  41. The perron-frobenius theorem: some of its applications. IEEE Signal Processing Magazine, 22(2):62–75, 2005.
  42. Deepinf: Social influence prediction with deep learning. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 2110–2119, 2018.
  43. Steffen Rendle. Item recommendation from implicit feedback. In Recommender Systems Handbook. Springer, 2022.
  44. Bpr: Bayesian personalized ranking from implicit feedback. In Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence, pages 452–461, 2009.
  45. Countering popularity bias by regularizing score differences. In Proceedings of the 16th ACM Conference on Recommender Systems, pages 145–155, 2022.
  46. A mathematical theory of semantic development in deep neural networks. Proceedings of the National Academy of Sciences, 116(23):11537–11546, 2019.
  47. Recommendations as treatments: Debiasing learning and evaluation. In international conference on machine learning, pages 1670–1679. PMLR, 2016.
  48. Session-based social recommendation via dynamic graph attention networks. In Proceedings of the Twelfth ACM international conference on web search and data mining, pages 555–563, 2019.
  49. Harald Steck. Item popularity and recommendation accuracy. In Proceedings of the fifth ACM conference on Recommender systems, pages 125–132, 2011.
  50. Spectral norm regularization for blind image deblurring. Symmetry, 13(10):1856, 2021.
  51. Long-tailed classification by keeping the good and removing the bad momentum causal effect. Advances in Neural Information Processing Systems, 33:1513–1524, 2020.
  52. Towards representation alignment and uniformity in collaborative filtering. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 1816–1825, 2022.
  53. Deconfounded recommendation for alleviating bias amplification. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pages 1717–1725, 2021.
  54. Model-agnostic counterfactual reasoning for eliminating popularity bias in recommender system. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pages 1791–1800, 2021.
  55. Self-supervised graph learning for recommendation. In Proceedings of the 44th international ACM SIGIR conference on research and development in information retrieval, pages 726–735, 2021.
  56. A survey on accuracy-oriented neural recommendation: From collaborative filtering to information-rich recommendation. TKDE, 2022.
  57. Neutralizing popularity bias in recommendation models. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 2623–2628, 2022.
  58. Spectral norm regularization for improving the generalizability of deep learning. arXiv preprint arXiv:1705.10941, 2017.
  59. Are graph augmentations necessary? simple graph contrastive learning for recommendation. In Proceedings of the 45th international ACM SIGIR conference on research and development in information retrieval, pages 1294–1303, 2022.
  60. Xsimgcl: Towards extremely simple graph contrastive learning for recommendation. IEEE Transactions on Knowledge and Data Engineering, 2023.
  61. Graph convolutional network for recommendation with low-pass collaborative filters. In International Conference on Machine Learning, pages 10936–10945. PMLR, 2020.
  62. Invariant collaborative filtering to popularity distribution shift. In Proceedings of the ACM Web Conference 2023, pages 1240–1251, 2023a.
  63. A model-agnostic popularity debias training framework for click-through rate prediction in recommender system. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 1760–1764, 2023.
  64. Addressing unmeasured confounding in comparative observational research. Pharmacoepidemiology and drug safety, 27(4):373–382, 2018.
  65. Causal intervention for leveraging popularity bias in recommendation. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 11–20, 2021.
  66. Mitigating the popularity bias of graph collaborative filtering: A dimensional collapse perspective. In Thirty-seventh Conference on Neural Information Processing Systems, 2023b.
  67. Popularity bias is not always evil: Disentangling benign and harmful bias for recommendation. IEEE Transactions on Knowledge and Data Engineering, 2022.
  68. Deep interest network for click-through rate prediction. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, pages 1059–1068, 2018.
  69. Adaptive popularity debiasing aggregator for graph collaborative filtering. pages 7–17, 2023.
  70. Popularity-opportunity bias in collaborative filtering. In Proceedings of the 14th ACM International Conference on Web Search and Data Mining, pages 85–93, 2021.
  71. Evolution of popularity bias: Empirical study and debiasing. arXiv preprint arXiv:2207.03372, 2022.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now