Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
158 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Improving Content Recommendation: Knowledge Graph-Based Semantic Contrastive Learning for Diversity and Cold-Start Users (2403.18667v1)

Published 27 Mar 2024 in cs.IR and cs.CL

Abstract: Addressing the challenges related to data sparsity, cold-start problems, and diversity in recommendation systems is both crucial and demanding. Many current solutions leverage knowledge graphs to tackle these issues by combining both item-based and user-item collaborative signals. A common trend in these approaches focuses on improving ranking performance at the cost of escalating model complexity, reducing diversity, and complicating the task. It is essential to provide recommendations that are both personalized and diverse, rather than solely relying on achieving high rank-based performance, such as Click-through Rate, Recall, etc. In this paper, we propose a hybrid multi-task learning approach, training on user-item and item-item interactions. We apply item-based contrastive learning on descriptive text, sampling positive and negative pairs based on item metadata. Our approach allows the model to better understand the relationships between entities within the knowledge graph by utilizing semantic information from text. It leads to more accurate, relevant, and diverse user recommendations and a benefit that extends even to cold-start users who have few interactions with items. We perform extensive experiments on two widely used datasets to validate the effectiveness of our approach. Our findings demonstrate that jointly training user-item interactions and item-based signals using synopsis text is highly effective. Furthermore, our results provide evidence that item-based contrastive learning enhances the quality of entity embeddings, as indicated by metrics such as uniformity and alignment.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (70)
  1. Gediminas Adomavicius and Alexander Tuzhilin. 2005. Toward the next generation of recommender systems: A survey of the state-of-the-art and possible extensions. IEEE transactions on knowledge and data engineering, 17(6):734–749.
  2. Ms marco: A human generated machine reading comprehension dataset. arXiv preprint arXiv:1611.09268.
  3. Universal sentence encoder for english. In Proceedings of the 2018 conference on empirical methods in natural language processing: system demonstrations, pages 169–174.
  4. On sampling strategies for neural network-based collaborative filtering. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 767–776.
  5. Wide & deep learning for recommender systems. In Proceedings of the 1st workshop on deep learning for recommender systems, pages 7–10.
  6. Anup Anand Deshmukh and Udhav Sethi. 2020. Ir-bert: leveraging bert for semantic search in background linking for news articles. arXiv preprint arXiv:2007.12603.
  7. Co-search: Covid-19 information retrieval with semantic search, question answering, and abstractive summarization. arXiv preprint arXiv:2006.09595.
  8. Kawin Ethayarajh. 2019. How contextual are contextualized word representations? comparing the geometry of bert, elmo, and gpt-2 embeddings. arXiv preprint arXiv:1909.00512.
  9. Simcse: Simple contrastive learning of sentence embeddings. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6894–6910.
  10. Learning dense representations for entity retrieval. arXiv preprint arXiv:1909.10506.
  11. Dimensionality reduction by learning an invariant mapping. In 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06), volume 2, pages 1735–1742. IEEE.
  12. Inductive representation learning on large graphs. Advances in neural information processing systems, 30.
  13. F Maxwell Harper and Joseph A Konstan. 2015. The movielens datasets: History and context. Acm transactions on interactive intelligent systems (tiis), 5(4):1–19.
  14. Neural collaborative filtering. In Proceedings of the 26th international conference on world wide web, pages 173–182.
  15. Efficient natural language response suggestion for smart reply. arXiv preprint arXiv:1705.00652.
  16. Thomas Hofmann. 2004. Latent semantic models for collaborative filtering. ACM Transactions on Information Systems (TOIS), 22(1):89–115.
  17. Improving sequential recommendation with knowledge-enhanced memory networks. In The 41st international ACM SIGIR conference on research & development in information retrieval, pages 505–514.
  18. Embarrassingly shallow auto-encoders for dynamic collaborative filtering. User Modeling and User-Adapted Interaction, 32(4):509–541.
  19. Dense passage retrieval for open-domain question answering. arXiv preprint arXiv:2004.04906.
  20. Jacob Devlin Ming-Wei Chang Kenton and Lee Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of NAACL-HLT, pages 4171–4186.
  21. Weakly labeled data augmentation for social media named entity recognition. Expert Systems with Applications, 209:118217.
  22. Siamese neural networks for one-shot image recognition. In ICML deep learning workshop, volume 2. Lille.
  23. Contrastive representation learning: A framework and review. IEEE Access, pages 193907–193934.
  24. Hkgcl: Hierarchical graph contrastive learning for multi-domain recommendation over knowledge graph. Expert Systems with Applications, 233:120963.
  25. Amazon. com recommendations: Item-to-item collaborative filtering. IEEE Internet computing, 7(1):76–80.
  26. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692.
  27. Enhancing recommendations with contrastive learning from collaborative knowledge graph. Neurocomputing, 523:103–115.
  28. Koji Miyahara and Michael J Pazzani. 2000. Collaborative filtering with the simple bayesian classifier. In Pacific Rim International conference on artificial intelligence, pages 679–689. Springer.
  29. Don’t judge a language model by its last layer: Contrastive learning with layer-wise attention pooling. arXiv preprint arXiv:2209.05972.
  30. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748.
  31. Minju Park and Kyogu Lee. 2022. Exploiting negative preference in content-based music recommendation with contrastive learning. In Proceedings of the 16th ACM Conference on Recommender Systems, pages 229–236.
  32. Don’t recommend the obvious: estimate probability ratios. In Proceedings of the 16th ACM Conference on Recommender Systems, pages 188–197.
  33. Gustavo Penha and Claudia Hauff. 2020. What does bert know about books, movies and music? probing bert for conversational recommendation. In Proceedings of the 14th ACM Conference on Recommender Systems, pages 388–397.
  34. A strong node classification baseline for temporal graphs. In Proceedings of the 2022 SIAM International Conference on Data Mining (SDM), pages 648–656. SIAM.
  35. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9.
  36. Exploring the limits of transfer learning with a unified text-to-text transformer. arXiv preprint arXiv:1910.10683.
  37. Bce4zsr: Bi-encoder empowered by teacher cross-encoder for zero-shot cold-start news recommendation. Information Processing & Management, 61(3):103686.
  38. Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. arXiv preprint arXiv:1908.10084.
  39. Jasson DM Rennie and Nathan Srebro. 2005. Fast maximum margin matrix factorization for collaborative prediction. In Proceedings of the 22nd international conference on Machine learning, pages 713–719.
  40. Learning to rank instant search results with multiple indices: A case study in search aggregation for entertainment. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 3412–3416.
  41. Item-based collaborative filtering recommendation algorithms. In Proceedings of the 10th international conference on World Wide Web, pages 285–295.
  42. Combining graph neural networks and sentence encoders for knowledge-aware recommendations. In Proceedings of the 31st ACM Conference on User Modeling, Adaptation and Personalization, pages 1–12.
  43. Maximum-margin matrix factorization. Advances in neural information processing systems, 17.
  44. Xiaoyuan Su and Taghi M Khoshgoftaar. 2006. Collaborative filtering for multi-class data using belief nets algorithms. In 2006 18th IEEE international conference on Tools with Artificial Intelligence (ICTAI’06), pages 497–504. IEEE.
  45. How to fine-tune bert for text classification? In Chinese Computational Linguistics: 18th China National Conference, CCL 2019, Kunming, China, October 18–20, 2019, Proceedings 18, pages 194–206. Springer.
  46. Investigation of various matrix factorization methods for large recommender systems. In Proceedings of the 2nd KDD Workshop on Large-Scale Recommender Systems and the Netflix Prize Competition, pages 1–8.
  47. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971.
  48. Lyle H Ungar and Dean P Foster. 1998. Clustering methods for collaborative filtering. In AAAI workshop on recommendation systems, volume 1, pages 114–129. Menlo Park, CA.
  49. Attention is all you need. Advances in neural information processing systems, 30.
  50. Mengting Wan and Julian McAuley. 2018. Item recommendation on monotonic behavior chains. In Proceedings of the 12th ACM conference on recommender systems, pages 86–94.
  51. Fine-grained spoiler detection from large-scale review corpora. arXiv preprint arXiv:1905.13416.
  52. Joint topic-semantic-aware social recommendation for online voting. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, pages 347–356.
  53. Ripplenet: Propagating user preferences on the knowledge graph for recommender systems. In Proceedings of the 27th ACM international conference on information and knowledge management, pages 417–426.
  54. Dkn: Deep knowledge-aware network for news recommendation. In Proceedings of the 2018 world wide web conference, pages 1835–1844.
  55. Knowledge-aware graph neural networks with label smoothness regularization for recommender systems. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pages 968–977.
  56. Knowledge graph convolutional networks for recommender systems. In The world wide web conference, pages 3307–3313.
  57. Tongzhou Wang and Phillip Isola. 2020. Understanding contrastive representation learning through alignment and uniformity on the hypersphere. In International Conference on Machine Learning, pages 9929–9939. PMLR.
  58. Kgat: Knowledge graph attention network for recommendation. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pages 950–958.
  59. Learning intents behind interactions with knowledge graph for recommendation. In Proceedings of the Web Conference 2021, pages 878–887.
  60. Ckan: collaborative knowledge-aware attentive network for recommender systems. In Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval, pages 219–228.
  61. Empowering news recommendation with pre-trained language models. In Proceedings of the 44th international ACM SIGIR conference on research and development in information retrieval, pages 1652–1656.
  62. A survey on large language models for recommendation. arXiv preprint arXiv:2305.19860.
  63. Unsupervised feature learning via non-parametric instance discrimination. In CVPR, pages 3733–3742.
  64. End-to-end open-domain question answering with bertserini. arXiv preprint arXiv:1902.01718.
  65. Composite re-ranking for efficient document search with bert. arXiv preprint arXiv:2103.06499.
  66. Probabilistic memory-based collaborative filtering. IEEE Transactions on Knowledge and Data Engineering, 16(1):56–69.
  67. Personalized entity recommendation: A heterogeneous information network approach. In Proceedings of the 7th ACM international conference on Web search and data mining, pages 283–292.
  68. Collaborative knowledge base embedding for recommender systems. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pages 353–362.
  69. Explainable recommendation: A survey and new perspectives. Foundations and Trends® in Information Retrieval, 14(1):1–101.
  70. Meta-graph based recommendation fusion over heterogeneous information networks. In Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining, pages 635–644.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now