Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
125 tokens/sec
GPT-4o
47 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

How Does Message Passing Improve Collaborative Filtering? (2404.08660v2)

Published 27 Mar 2024 in cs.IR and cs.LG

Abstract: Collaborative filtering (CF) has exhibited prominent results for recommender systems and been broadly utilized for real-world applications. A branch of research enhances CF methods by message passing used in graph neural networks, due to its strong capabilities of extracting knowledge from graph-structured data, like user-item bipartite graphs that naturally exist in CF. They assume that message passing helps CF methods in a manner akin to its benefits for graph-based learning tasks in general. However, even though message passing empirically improves CF, whether or not this assumption is correct still needs verification. To address this gap, we formally investigate why message passing helps CF from multiple perspectives and show that many assumptions made by previous works are not entirely accurate. With our curated ablation studies and theoretical analyses, we discover that (1) message passing improves the CF performance primarily by additional representations passed from neighbors during the forward pass instead of additional gradient updates to neighbor representations during the model back-propagation and (ii) message passing usually helps low-degree nodes more than high-degree nodes. Utilizing these novel findings, we present Test-time Aggregation for CF, namely TAG-CF, a test-time augmentation framework that only conducts message passing once at inference time. The key novelty of TAG-CF is that it effectively utilizes graph knowledge while circumventing most of notorious computational overheads of message passing. Besides, TAG-CF is extremely versatile can be used as a plug-and-play module to enhance representations trained by different CF supervision signals. Evaluated on six datasets, TAG-CF consistently improves the recommendation performance of CF methods without graph by up to 39.2% on cold users and 31.7% on all users, with little to no extra computational overheads.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (67)
  1. Lightgcl: Simple yet effective graph contrastive learning for recommendation. In Procs. of ICLR (2023).
  2. Sequential recommendation with graph neural networks. In Procs. of SIGIR (2021).
  3. Efficient neural matrix factorization without sampling for recommendation. ACM Transactions on Information Systems (TOIS) (2020).
  4. Bias and debias in recommender system: A survey and future directions. ACM Transactions on Information Systems 41, 3 (2023), 1–39.
  5. Friendship and mobility: user movement in location-based social networks. In Procs of SIGKDD (2011).
  6. Learning parametrised graph shift operators. In Procs of ICLR (2021).
  7. Learning laplacian matrix in smooth graph signal representations. IEEE Transactions on Signal Processing (2016).
  8. Learning graphs from data: A signal representation perspective. IEEE Signal Processing Magazine (2019).
  9. Heterogeneous temporal graph neural network. In Procs. of SDM (2022).
  10. Graph neural networks for recommender system. In Procs. of WSDM (2022).
  11. A survey of graph neural networks for recommender systems: Challenges, methods, and directions. ACM Transactions on Recommender Systems (2023).
  12. Neural message passing for quantum chemistry. In Procs. of ICML (2017).
  13. The netflix recommender system: Algorithms, business value, and innovation. ACM Transactions on Management Information Systems (TMIS) (2015).
  14. Linkless link prediction via relational distillation. In International Conference on Machine Learning (2023), PMLR, pp. 12012–12033.
  15. Node duplication improves cold-start link prediction. arXiv preprint arXiv:2402.09711 (2024).
  16. Inductive representation learning on large graphs. In Procs. of NeurIPS (2017).
  17. The movielens datasets: History and context. Acm transactions on interactive intelligent systems (tiis) (2015).
  18. Lightgcn: Simplifying and powering graph convolution network for recommendation. In Procs. of SIGIR (2020).
  19. Neural collaborative filtering. In Procs. of WWW (2017).
  20. Tuneup: A training strategy for improving generalization of graph neural networks. arXiv (2022).
  21. Let graph be the go board: gradient-free node injection attack for graph neural networks via reinforcement learning. In Procs. of AAAI (2023).
  22. Multi-task self-supervised graph neural networks enable stronger task generalization. In Procs. of ICLR (2023).
  23. Graphpatcher: Mitigating degree bias for graph neural networks via test-time augmentation. In Procs. of NeurIPS (2023).
  24. Kaggle. Anime Recommendatiosn Database. https://www.kaggle.com/datasets/CooperUnion/anime-recommendations-database, 2023. [Online; accessed June-2023].
  25. Semi-supervised classification with graph convolutional networks. In Procs. of ICLR (2016).
  26. Matrix factorization techniques for recommender systems. Computer (2009).
  27. Advances in collaborative filtering. Recommender systems handbook (2021), 91–142.
  28. Improving graph collaborative filtering with neighborhood-enriched contrastive learning. In Procs of WWW (2022).
  29. Tail-gnn: Tail-node graph neural networks. In Procs. of SIGKDD (2021).
  30. Is homophily a necessity for graph neural networks? In Procs. of ICLR (2022).
  31. A unified view on graph neural networks as graph signal denoising. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management (2021), pp. 1202–1211.
  32. Ultragcn: ultra simplification of graph convolutional networks for recommendation. In Procs. of CIKM (2021).
  33. Addressing complex and subjective product-related queries with customer reviews. In Procs. of WWW (2016).
  34. Graph neural architecture search: A survey. Tsinghua Science and Technology (2021).
  35. Pinnersage: Multi-modal user embedding framework for recommendations at pinterest. In Procs. of SIGKDD (2020).
  36. Svd-gcn: A simplified graph convolution paradigm for recommendation. In Procs of CIKM (2022).
  37. Bpr: Bayesian personalized ranking from implicit feedback. In UAI (2009).
  38. Graph neural networks for friend ranking in large-scale social platforms. In Proceedings of the Web Conference 2021 (2021), pp. 2535–2546.
  39. Recommender systems in e-commerce. In Procs. of ACM conference on Electronic commerce (1999).
  40. How powerful is graph convolution for recommendation? In Procs. of CIKM (2021).
  41. Heterogeneous graph neural network for recommendation. arXiv preprint arXiv:2009.00799 (2020).
  42. A survey of collaborative filtering techniques. Advances in artificial intelligence (2009).
  43. Investigating and mitigating degree-related biases in graph convoltuional networks. In Procs. of CIKM (2020).
  44. Deep content-based music recommendation. In Procs. of NeurIPS (2013).
  45. Graph attention networks. In Procs. of ICLR (2017).
  46. Towards representation alignment and uniformity in collaborative filtering. In Procs. of SIGKDD (2022).
  47. Collaborative deep learning for recommender systems. In Procs. of SIGKDD (2015).
  48. Dcn v2: Improved deep & cross network and practical lessons for web-scale learning to rank systems. In Procs. of WWW (2021).
  49. Neural graph collaborative filtering. In Procs. of SIGIR (2019).
  50. Disentangled graph collaborative filtering. In Procs. of SIGIR (2020).
  51. Bag of tricks for node classification with graph neural networks. arXiv preprint arXiv:2103.13355 (2021).
  52. Contrastive learning for cold-start recommendation. In Proceedings of the 29th ACM International Conference on Multimedia (2021), pp. 5382–5390.
  53. Self-supervised graph learning for recommendation. In Procs. of SIGIR (2021).
  54. Graph neural networks in recommender systems: a survey. ACM Computing Surveys 55, 5 (2022), 1–37.
  55. Graph-less collaborative filtering. In Procs. of WWW (2023).
  56. Graph meta network for multi-behavior recommendation. In Procs. of SIGIR (2021).
  57. How powerful are graph neural networks? In Procs. of ICLR (2018).
  58. Representation learning on graphs with jumping knowledge networks. In Procs. of ICML (2018).
  59. Yelp. Yelp Open Dataset. https://www.yelp.com/dataset, 2023. [Online; accessed June-2023].
  60. Graph convolutional neural networks for web-scale recommender systems. In Procs. of SIGKDD (2018).
  61. Self-supervised multi-channel hypergraph convolutional network for social recommendation. In Procs. of WWW (2021).
  62. Are graph augmentations necessary? simple graph contrastive learning for recommendation. In Procs. of SIGIR (2022).
  63. Decoupling the depth and scope of graph neural networks. Advances in Neural Information Processing Systems 34 (2021), 19665–19679.
  64. Graph-less neural networks: Teaching old mlps new tricks via distillation. arXiv preprint arXiv:2110.08727 (2021).
  65. Deep learning based recommender system: A survey and new perspectives. ACM computing surveys (CSUR) (2019).
  66. Recbole 2.0: towards a more up-to-date recommendation library. In Procs. of CIKM (2022).
  67. Interpreting and unifying graph neural networks with an optimization framework. In Procs. of WWW (2021).
Citations (3)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets