Self-supervised Contrastive Learning for Implicit Collaborative Filtering (2403.07265v1)
Abstract: Contrastive learning-based recommendation algorithms have significantly advanced the field of self-supervised recommendation, particularly with BPR as a representative ranking prediction task that dominates implicit collaborative filtering. However, the presence of false-positive and false-negative examples in recommendation systems hampers accurate preference learning. In this study, we propose a simple self-supervised contrastive learning framework that leverages positive feature augmentation and negative label augmentation to improve the self-supervisory signal. Theoretical analysis demonstrates that our learning method is equivalent to maximizing the likelihood estimation with latent variables representing user interest centers. Additionally, we establish an efficient negative label augmentation technique that samples unlabeled examples with a probability linearly dependent on their relative ranking positions, enabling efficient augmentation in constant time complexity. Through validation on multiple datasets, we illustrate the significant improvements our method achieves over the widely used BPR optimization objective while maintaining comparable runtime.
- Learning representations by maximizing mutual information across views. (2019).
- Contrastive curriculum learning for sequential user behavior modeling via data augmentation. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management. 3737–3746.
- LightGCL: Simple Yet Effective Graph Contrastive Learning for Recommendation. International Conference on Learning Representations (2023).
- Big self-supervised models are strong semi-supervised learners. Proceedings of the International Conference on Neural Information Processing Systems 33 (2020), 22243–22255.
- Micro-Supervised Disturbance Learning: A Perspective of Representation Probability Distribution. IEEE Transactions on Pattern Analysis and Machine Intelligence 45 (2023), 7542–7558.
- Debiased Contrastive Learning. In Proceedings of the International Conference on Neural Information Processing Systems. 8765–8775.
- Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society: Series B (Methodological) 39, 1 (1977), 1–22.
- Reinforced Negative Sampling for Recommendation with Exposure Data. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence. 2230–2236.
- Simplify and Robustify Negative Sampling for Implicit Collaborative Filtering. In Proceedings of the International Conference on Neural Information Processing Systems.
- Discriminative unsupervised feature learning with convolutional neural networks. In Proceedings of the International Conference on Neural Information Processing Systems. 766–774.
- Recommendation as language processing (rlp): A unified pretrain, personalized prompt & predict paradigm (p5). In Proceedings of the 16th ACM Conference on Recommender Systems. 299–315.
- Bootstrap Your Own Latent a New Approach to Self-Supervised Learning. In Proceedings of the 34th International Conference on Neural Information Processing Systems. 21271–21284.
- Michael Gutmann and Aapo Hyvärinen. 2010. Noise-contrastive estimation: A new estimation principle for unnormalized statistical models. In Proceedings of the thirteenth international conference on artificial intelligence and statistics. 297–304.
- Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF conference on Computer Vision and Pattern Recognition. 9729–9738.
- Learning deep representations by mutual information estimation and maximization. arXiv preprint arXiv:1808.06670 (2018).
- Unsupervised deep learning by neighbourhood discovery. In International Conference on Machine Learning. 2849–2858.
- Siddharth Joshi and Baharan Mirzasoleiman. 2023. Data-Efficient Contrastive Self-supervised Learning: Easy Examples Contribute the Most. arXiv preprint arXiv:2302.09195 (2023).
- Matrix factorization techniques for recommender systems. Computer 42, 8 (2009), 30–37.
- Prototypical Contrastive Learning of Unsupervised Representations. In International Conference on Learning Representations. 353–356.
- Improving graph collaborative filtering with neighborhood-enriched contrastive learning. In Proceedings of the ACM Web Conference 2022. 2320–2329.
- Bin Liu and Bang Wang. 2023. Bayesian Negative Sampling for Recommendation. In 2023 IEEE 39th International Conference on Data Engineering (ICDE). IEEE, 749–761.
- Self-supervised learning: Generative or contrastive. IEEE Transactions on Knowledge and Data Engineering (2021).
- Contrastive learning for recommender system. arXiv preprint arXiv:2101.01317 (2021).
- Segclip: Patch aggregation with learnable centers for open-vocabulary semantic segmentation. In ICML. 23033–23044.
- Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748 (2018).
- Contrastive learning for representation degeneration problem in sequential recommendation. In Proceedings of the fifteenth ACM international conference on web search and data mining. 813–823.
- Steffen Rendle and Christoph Freudenthaler. 2014. Improving pairwise learning for item recommendation from implicit feedback. In Proceedings of the 7th ACM international conference on Web Search and Data Mining. 273–282.
- BPR: Bayesian Personalized Ranking from Implicit Feedback. In UAI 2009, Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence, Montreal, QC, Canada, June 18-21, 2009. 452–461.
- Contrastive Learning with Hard Negative Samples. In International Conference on Learning Representations.
- BERT4Rec: Sequential recommendation with bidirectional encoder representations from transformer. In Proceedings of the 28th ACM international conference on information and knowledge management. 1441–1450.
- Curriculum pre-training heterogeneous subgraph transformer for top-n recommendation. ACM Transactions on Information Systems 41, 1 (2023), 1–28.
- Tongzhou Wang and Phillip Isola. 2020. Understanding contrastive representation learning through alignment and uniformity on the hypersphere. In International Conference on Machine Learning. 9929–9939.
- Denoising implicit feedback for recommendation. In Proceedings of the 14th ACM international conference on web search and data mining. 373–381.
- Neural Graph Collaborative Filtering. In Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval. 2344–2353.
- Graph Trend Networks for Recommendations. In Proceedings of the ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 12.
- Self-Supervised Graph Learning for Recommendation. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. 726–735.
- On the Effectiveness of Sampled Softmax Loss for Item Recommendation. arXiv preprint arXiv:2201.02327 (2022).
- Self-Supervised Learning on Graphs: Contrastive, Generative, or Predictive. IEEE Transactions on Knowledge and Data Engineering 35, 4 (2023), 4216–4235.
- Unsupervised feature learning via non-parametric instance discrimination. In Proceedings of the IEEE/CVF conference on Computer Vision and Pattern Recognition. 3733–3742.
- LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation.. In Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval. 10.
- Knowledge Graph Contrastive Learning for Recommendation. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. 1434–1443.
- Understanding negative sampling in graph representation learning. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 1666–1676.
- XSimGCL: Towards extremely simple graph contrastive learning for recommendation. IEEE Transactions on Knowledge and Data Engineering (2023).
- Self-Supervised Learning for Recommender Systems: A Survey. IEEE Transactions on Knowledge and Data Engineering (2023), 1–20.
- Understanding deep learning (still) requires rethinking generalization. Commun. ACM 64, 3 (2021), 107–115.
- Optimizing Top-n Collaborative Filtering via Dynamic Negative Item Sampling. In Proceedings of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval. 785–788.
- Zizhuo Zhang and Bang Wang. 2023. Prompt Learning for News Recommendation. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval. 227–237.
- Bootstrap Latent Representations for Multi-Modal Recommendation. In Proceedings of the ACM Web Conference 2023. 845–854.