Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
110 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Prototypical Contrastive Learning through Alignment and Uniformity for Recommendation (2402.02079v1)

Published 3 Feb 2024 in cs.IR and cs.AI

Abstract: Graph Collaborative Filtering (GCF), one of the most widely adopted recommendation system methods, effectively captures intricate relationships between user and item interactions. Graph Contrastive Learning (GCL) based GCF has gained significant attention as it leverages self-supervised techniques to extract valuable signals from real-world scenarios. However, many methods usually learn the instances of discrimination tasks that involve the construction of contrastive pairs through random sampling. GCL approaches suffer from sampling bias issues, where the negatives might have a semantic structure similar to that of the positives, thus leading to a loss of effective feature representation. To address these problems, we present the \underline{Proto}typical contrastive learning through \underline{A}lignment and \underline{U}niformity for recommendation, which is called \textbf{ProtoAU}. Specifically, we first propose prototypes (cluster centroids) as a latent space to ensure consistency across different augmentations from the origin graph, aiming to eliminate the need for random sampling of contrastive pairs. Furthermore, the absence of explicit negatives means that directly optimizing the consistency loss between instance and prototype could easily result in dimensional collapse issues. Therefore, we propose aligning and maintaining uniformity in the prototypes of users and items as optimization objectives to prevent falling into trivial solutions. Finally, we conduct extensive experiments on four datasets and evaluate their performance on the task of link prediction. Experimental results demonstrate that the proposed ProtoAU outperforms other representative methods. The source codes of our proposed ProtoAU are available at \url{https://github.com/oceanlvr/ProtoAU}.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (31)
  1. J. L. Herlocker, J. A. Konstan, L. G. Terveen, and J. Riedl, “Evaluating collaborative filtering recommender systems,” ACM Transactions on Information Systems, vol. 22, no. 1, pp. 5–53, 2004.
  2. Y. Zhao, F. Jiang, and J. Wang, “Mfgcl: Light graph contrast learning with multi-feature information for recommendation,” in Proceedings of the 33rd International Joint Conference on Neural Networks (IJCNN).   Coast, Australia: IEEE, 2023, pp. 1–8.
  3. R. Gao, J. Liu, Y. Yu, D. Liu, X. Shao, and Z. Ye, “Gated dual hypergraph convolutional networks for recommendation with self-supervised learning,” in Proceedings of the 32nd International Joint Conference on Neural Networks (IJCNN).   Padua,Italy: IEEE, 2022, pp. 1–8.
  4. R. van den Berg, T. N. Kipf, and M. Welling, “Graph convolutional matrix completion,” Knowledge Discovery and Data Mining, 2017.
  5. W. Hamilton, Z. Ying, and J. Leskovec, “Inductive representation learning on large graphs,” Advances in neural information processing systems, vol. 30, p. 1025–1035, 2017.
  6. S. Mandal and A. Maiti, “Graph neural networks for heterogeneous trust based social recommendation,” in Proceedings of the 31st International Joint Conference on Neural Networks (IJCNN).   Shenzhen, China: IEEE, 2021, pp. 1–8.
  7. J. Wu, X. Wang, F. Feng, X. He, L. Chen, J. Lian, and X. Xie, “Self-supervised graph learning for recommendation,” in Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR).   Virtual Event: ACM, 2021, pp. 726–735.
  8. J. Yu, H. Yin, X. Xia, T. Chen, J. Li, and Z. Huang, “Self-supervised learning for recommender systems: A survey,” IEEE Transactions on Knowledge and Data Engineering, vol. 36, no. 1, pp. 335–355, 2024.
  9. Z. Lin, C. Tian, Y. Hou, and W. X. Zhao, “Improving graph collaborative filtering with neighborhood-enriched contrastive learning,” in Proceedings of the ACM Web Conference 2022 (WWW).   Virtual Event: ACM, 2022, pp. 2320–2329.
  10. J. Yu, H. Yin, M. Gao, X. Xia, X. Zhang, and N. Q. V. Hung, “Socially-aware self-supervised tri-training for recommendation,” in Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD).   Virtual Event: ACM, 2021, pp. 2084–2092.
  11. J. Cao, X. Lin, S. Guo, L. Liu, T. Liu, and B. Wang, “Bipartite graph embedding via mutual information maximization,” in Proceedings of the 14th ACM international conference on web search and data mining (WSDM).   ACM, 2021, pp. 635–643.
  12. J. Yu, H. Yin, J. Li, Q. Wang, N. Q. V. Hung, and X. Zhang, “Self-supervised multi-channel hypergraph convolutional network for social recommendation,” in Proceedings of the web conference 2021 (www).   Ljubljana, Slovenia: ACM / IW3C2, 2021, pp. 413–424.
  13. P. Bachman, R. D. Hjelm, and W. Buchwalter, “Learning representations by maximizing mutual information across views,” Advances in neural information processing systems, vol. 32, no. 1392, pp. 15 535–15 545, 2019.
  14. K. Zhou, B. Zhang, W. X. Zhao, and J.-R. Wen, “Debiased contrastive learning of unsupervised sentence representations,” in Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (ACL).   Dublin, Ireland: Association for Computational Linguistics, 2022, pp. 6120–6130.
  15. J. D. Robinson, C.-Y. Chuang, S. Sra, and S. Jegelka, “Contrastive learning with hard negative samples,” in Proceedings of the 9th International Conference on Learning Representations (ICLR).   Virtual Event: OpenReview.net, 2021.
  16. C. Wang, Y. Yu, W. Ma, M. Zhang, C. Chen, Y. Liu, and S. Ma, “Towards representation alignment and uniformity in collaborative filtering,” in Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD).   Washington, District of Columbia: ACM, 2022, pp. 1816–1825.
  17. T. Wang and P. Isola, “Understanding contrastive representation learning through alignment and uniformity on the hypersphere,” in Proceedings of the 37th International Conference on Machine Learning (ICML).   Virtual Event: PMLR, 2020, pp. 9929–9939.
  18. H. Chen, V. Lai, H. Jin, Z. Jiang, M. Das, and X. Hu, “Towards mitigating dimensional collapse of representations in collaborative filtering,” arXiv preprint arXiv:2312.17468, 2023.
  19. J. Yu, H. Yin, X. Xia, T. Chen, L. Cui, and Q. V. H. Nguyen, “Are graph augmentations necessary?: Simple graph contrastive learning for recommendation,” in Proceeedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR).   ACM, 2022, pp. 1294–1303.
  20. T. Yao, X. Yi, D. Z. Cheng, F. X. Yu, T. Chen, A. K. Menon, L. Hong, E. H. Chi, S. Tjoa, J. J. Kang, and E. Ettinger, “Self-supervised learning for large-scale item recommendations,” in Proceedings of the 30th ACM International Conference on Information and Knowledge Management (CIKM).   Virtual Event: Association for Computing Machinery, 2021, pp. 4321–4330.
  21. A. v. d. Oord, Y. Li, and O. Vinyals, “Representation learning with contrastive predictive coding,” arXiv preprint arXiv:1807.03748, 2018.
  22. X. Cai, C. Huang, L. Xia, and X. Ren, “Lightgcl: Simple yet effective graph contrastive learning for recommendation,” in Proceedings of the 11th International Conference on Learning Representations (ICLR).   Kigali, Rwanda: OpenReview.net, 2023.
  23. X. He, K. Deng, X. Wang, Y. Li, Y. Zhang, and M. Wang, “Lightgcn: Simplifying and powering graph convolution network for recommendation,” in Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval (SIGIR).   Virtual Event: ACM, 2020, pp. 639–648.
  24. S. Rendle, C. Freudenthaler, Z. Gantner, and L. Schmidt-Thieme, “Bpr: Bayesian personalized ranking from implicit feedback,” in Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence (UAI).   Montreal,Canada: AUAI Press, 2009, pp. 452–461.
  25. T. Hua, W. Wang, Z. Xue, S. Ren, Y. Wang, and H. Zhao, “On feature decorrelation in self-supervised learning,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV).   Montreal, Canada: IEEE, 2021, pp. 9578–9588.
  26. M. Cuturi, “Sinkhorn distances: Lightspeed computation of optimal transport,” Advances in neural information processing systems, vol. 26, pp. 2292–2300, 2013.
  27. J. J. McAuley, C. Targett, Q. Shi, and A. van den Hengel, “Image-based recommendations on styles and substitutes,” in Proceedings of the 38th international ACM SIGIR conference on research and development in information retrieval (SIGIR).   Santiago, Chile: ACM, 2015, pp. 43–52.
  28. W. Chen, P. Huang, J. Xu, X. Guo, C. Guo, F. Sun, C. Li, A. Pfadler, H. Zhao, and B. Zhao, “Pog: Personalized outfit generation for fashion recommendation at alibaba ifashion,” in Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD).   Anchorage, Alaska: ACM, 2019, pp. 2662–2670.
  29. F. M. Harper and J. A. Konstan, “The movielens datasets: History and context,” Acm transactions on interactive intelligent systems, vol. 5, no. 4, pp. 1–19, 2015.
  30. X. Wang, X. He, M. Wang, F. Feng, and T. Chua, “Neural graph collaborative filtering,” in Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR).   Paris,France: ACM, 2019, pp. 165–174.
  31. L. Van der Maaten and G. Hinton, “Visualizing data using t-sne.” Journal of machine learning research, vol. 9, no. 11, 2008.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (4)
  1. Yangxun Ou (1 paper)
  2. Lei Chen (485 papers)
  3. Fenglin Pan (1 paper)
  4. Yupeng Wu (7 papers)

Summary

We haven't generated a summary for this paper yet.

Github Logo Streamline Icon: https://streamlinehq.com