Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Is Contrastive Learning Necessary? A Study of Data Augmentation vs Contrastive Learning in Sequential Recommendation (2403.11136v1)

Published 17 Mar 2024 in cs.IR

Abstract: Sequential recommender systems (SRS) are designed to predict users' future behaviors based on their historical interaction data. Recent research has increasingly utilized contrastive learning (CL) to leverage unsupervised signals to alleviate the data sparsity issue in SRS. In general, CL-based SRS first augments the raw sequential interaction data by using data augmentation strategies and employs a contrastive training scheme to enforce the representations of those sequences from the same raw interaction data to be similar. Despite the growing popularity of CL, data augmentation, as a basic component of CL, has not received sufficient attention. This raises the question: Is it possible to achieve superior recommendation results solely through data augmentation? To answer this question, we benchmark eight widely used data augmentation strategies, as well as state-of-the-art CL-based SRS methods, on four real-world datasets under both warm- and cold-start settings. Intriguingly, the conclusion drawn from our study is that, certain data augmentation strategies can achieve similar or even superior performance compared with some CL-based methods, demonstrating the potential to significantly alleviate the data sparsity issue with fewer computational overhead. We hope that our study can further inspire more fundamental studies on the key functional components of complex CL techniques. Our processed datasets and codes are available at https://github.com/AIM-SE/DA4Rec.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (64)
  1. Controllable multi-interest framework for recommendation. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2942–2951.
  2. Sequential Recommendation with Graph Neural Networks. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’21). 378–387.
  3. Behavior sequence transformer for e-commerce recommendation in alibaba. In Proceedings of the 1st International Workshop on Deep Learning Practice for High-Dimensional Sparse Data. 1–4.
  4. Intent Contrastive Learning for Sequential Recommendation. In WWW. ACM, 2172–2182.
  5. A Case Study on Sampling Strategies for Evaluating Neural Sequential Item Recommendation Models. In Fifteenth ACM Conference on Recommender Systems. 505–514.
  6. Uniform Sequence Better: Time Interval Aware Data Augmentation for Sequential Recommendation. arXiv preprint arXiv:2212.08262 (2022).
  7. A unified personalized video recommendation via dynamic recurrent neural networks. In Proceedings of the 25th ACM international conference on Multimedia. 127–135.
  8. Evolutionary preference learning via graph nested gru ode for session-based recommendation. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management. 624–634.
  9. Contextual and sequential user embeddings for large-scale music recommendation. In Proceedings of the 14th ACM Conference on Recommender Systems. 53–62.
  10. Ruining He and Julian McAuley. 2016. Fusing Similarity Models with Markov Chains for Sparse Sequential Recommendation. In 2016 IEEE 16th International Conference on Data Mining (ICDM). 191–200.
  11. Session-based Recommendations with Recurrent Neural Networks. In ICLR.
  12. MixGCF: An Improved Training Method for Graph Neural Network-Based Recommender Systems. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD ’21). 665–674.
  13. Csan: Contextual self-attention network for user sequential recommendation. In Proceedings of the 26th ACM international conference on Multimedia. 447–455.
  14. Sequential recommendation with relation-aware kernelized self-attention. In AAAI 2020 - 34th AAAI Conference on Artificial Intelligence (AAAI 2020 - 34th AAAI Conference on Artificial Intelligence). 4304–4311.
  15. Sequential recommendation with bidirectional chronological augmentation of transformer. arXiv preprint arXiv:2112.06460 (2021).
  16. AdaMCT: adaptive mixture of CNN-transformer for sequential recommendation. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management. 976–986.
  17. Personalized travel sequence recommendation on multi-source big social media. IEEE Transactions on Big Data 2, 1 (2016), 43–56.
  18. Wang-Cheng Kang and Julian McAuley. 2018a. Self-attentive sequential recommendation. In ICDM. IEEE, 197–206.
  19. Wang-Cheng Kang and Julian McAuley. 2018b. Self-attentive sequential recommendation. In 2018 IEEE international conference on data mining (ICDM). IEEE, 197–206.
  20. Walid Krichene and Steffen Rendle. 2020. On sampled metrics for item recommendation. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 1748–1757.
  21. Bootstrapping User and Item Representations for One-Class Collaborative Filtering. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’21). 317–326.
  22. Semi: A sequential multi-modal information transfer network for e-commerce micro-video recommendations. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 3161–3171.
  23. Practical Content-aware Session-based Recommendation: Deep Retrieve then Shallow Rank. In Amazon KDD Cup 2023 Workshop.
  24. Neural rating regression with abstractive tips generation for recommendation. In Proceedings of the 40th International ACM SIGIR conference on Research and Development in Information Retrieval. 345–354.
  25. Is chatgpt a good recommender? a preliminary study. arXiv preprint arXiv:2304.10149v2 (2023).
  26. Llmrec: Benchmarking large language models on recommendation task. arXiv preprint arXiv:2308.12241 (2023).
  27. Contrastive Self-supervised Sequential Recommendation with Robust Augmentation. CoRR abs/2108.06479 (2021).
  28. Augmenting Sequential Recommendation with Pseudo-Prior Items via Reversely Pre-Training Transformer. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’21). 1608–1612.
  29. Image-Based Recommendations on Styles and Substitutes. In Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval, Santiago, Chile, August 9-13, 2015, Ricardo Baeza-Yates, Mounia Lalmas, Alistair Moffat, and Berthier A. Ribeiro-Neto (Eds.). ACM, 43–52. https://doi.org/10.1145/2766462.2767755
  30. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748 (2018).
  31. Online learning to rank for sequential music recommendation. In Proceedings of the 13th ACM Conference on Recommender Systems. 237–245.
  32. Aleksandr Petrov and Craig Macdonald. 2022. Effective and Efficient Training for Sequential Recommendation Using Recency Sampling. In Proceedings of the 16th ACM Conference on Recommender Systems (RecSys ’22). 81–91.
  33. Contrastive Learning for Representation Degeneration Problem in Sequential Recommendation. In WSDM. ACM, 813–823.
  34. Factorizing Personalized Markov Chains for Next-Basket Recommendation (WWW ’10). 811–820.
  35. Jooeun Song and Bongwon Suh. 2022. Data Augmentation Strategies for Improving Sequential Recommender Systems. ArXiv abs/2203.14037 (2022).
  36. Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning research 15, 1 (2014), 1929–1958.
  37. BERT4Rec: Sequential recommendation with bidirectional encoder representations from transformer. In Proceedings of the 28th ACM international conference on information and knowledge management. 1441–1450.
  38. Improved Recurrent Neural Networks for Session-Based Recommendations. In Proceedings of the 1st Workshop on Deep Learning for Recommender Systems (DLRS 2016). 17–22.
  39. Jiaxi Tang and Ke Wang. 2018a. Personalized top-n sequential recommendation via convolutional sequence embedding. In WSDM. 565–573.
  40. Jiaxi Tang and Ke Wang. 2018b. Personalized top-n sequential recommendation via convolutional sequence embedding. In Proceedings of the eleventh ACM international conference on web search and data mining. 565–573.
  41. Modelling local and global dependencies for next-item recommendations. In International Conference on Web Information Systems Engineering. Springer, 285–300.
  42. Exploiting intra-and inter-session dependencies for session-based recommendations. World Wide Web 25, 1 (2022), 425–443.
  43. Enhancing Collaborative Filtering with Generative Augmentation. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD ’19). 548–556.
  44. Sequential recommender systems: challenges, progress and prospects. In Proceedings of the 28th International Joint Conference on Artificial Intelligence. AAAI Press, 6332–6338.
  45. Intention nets: psychology-inspired user choice behavior modeling for next-basket prediction. In 34th AAAI Conference on Artificial Intelligence, AAAI 2020. Association for the Advancement of Artificial Intelligence, 6259–6266.
  46. Sequential/Session-based Recommendations: Challenges, Approaches, Applications and Opportunities. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. 3425–3428.
  47. Counterfactual Data-Augmented Sequential Recommendation. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’21). 347–356.
  48. Contrastive learning for sequential recommendation. arXiv preprint arXiv:2010.14395 (2020).
  49. Rethinking Multi-Interest Learning for Candidate Matching in Recommender Systems. arXiv preprint arXiv:2302.14532 (2023).
  50. Decoupled Side Information Fusion for Sequential Recommendation. In International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR).
  51. Self-Supervised Reinforcement Learning for Recommender Systems. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’20). 931–940.
  52. Multi-order attentive ranking model for sequential recommendation. In Proceedings of the AAAI conference on artificial intelligence, Vol. 33. 5709–5716.
  53. A simple convolutional generative network for next item recommendation. In Proceedings of the twelfth ACM international conference on web search and data mining. 582–590.
  54. ICAI-SR: Item Categorical Attribute Integrated Sequential Recommendation. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. 1687–1691.
  55. Efficiently leveraging multi-level user intent for session-based recommendation via atten-mixer network. In Proceedings of the Sixteenth ACM International Conference on Web Search and Data Mining. 168–176.
  56. Peiyan Zhang and Sunghun Kim. 2023. A Survey on Incremental Update for Neural Recommender Systems. arXiv preprint arXiv:2303.02851 (2023).
  57. Can Transformer and GNN Help Each Other? arXiv preprint arXiv:2308.14355 (2023).
  58. CauseRec: Counterfactual User Sequence Synthesis for Sequential Recommendation. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’21). 367–377.
  59. RecBole 2.0: Towards a More Up-to-Date Recommendation Library. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management. 4722–4726.
  60. S3-rec: Self-supervised learning for sequential recommendation with mutual information maximization. In Proceedings of the 29th ACM international conference on information & knowledge management. 1893–1902.
  61. Filter-Enhanced MLP is All You Need for Sequential Recommendation. In Proceedings of the ACM Web Conference 2022 (WWW ’22). 2388–2399.
  62. Exploring recommendation capabilities of gpt-4v (ision): A preliminary case study. arXiv preprint arXiv:2311.04199 (2023).
  63. Equivariant contrastive learning for sequential recommendation. In Proceedings of the 17th ACM Conference on Recommender Systems. 129–140.
  64. Attention Calibration for Transformer-based Sequential Recommendation. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management. 3595–3605.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (7)
  1. Peilin Zhou (34 papers)
  2. You-Liang Huang (2 papers)
  3. Yueqi Xie (22 papers)
  4. Jingqi Gao (4 papers)
  5. Shoujin Wang (40 papers)
  6. Jae Boum Kim (4 papers)
  7. Sunghun Kim (44 papers)
Citations (3)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now