Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
102 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

A Pre-trained Sequential Recommendation Framework: Popularity Dynamics for Zero-shot Transfer (2401.01497v4)

Published 3 Jan 2024 in cs.IR

Abstract: Sequential recommenders are crucial to the success of online applications, \eg e-commerce, video streaming, and social media. While model architectures continue to improve, for every new application domain, we still have to train a new model from scratch for high quality recommendations. On the other hand, pre-trained language and vision models have shown great success in zero-shot or few-shot adaptation to new application domains. Inspired by the success of pre-trained models in peer AI fields, we propose a novel pre-trained sequential recommendation framework: PrepRec. We learn universal item representations by modeling item popularity dynamics. Through extensive experiments on five real-world datasets, we show that PrepRec, without any auxiliary information, can not only zero-shot transfer to a new domain, but achieve competitive performance compared to state-of-the-art sequential recommender models with only a fraction of the model size. In addition, with a simple post-hoc interpolation, PrepRec can improve the performance of existing sequential recommenders on average by 13.8\% in Recall@10 and 29.5% in NDCG@10. We provide an anonymized implementation of PrepRec at https://anonymous.4open.science/r/PrepRec--2F60/

Definition Search Book Streamline Icon: https://streamlinehq.com
References (63)
  1. Layer normalization. arXiv preprint arXiv:1607.06450 (2016).
  2. Albert-Laszlo Barabasi. 2005. The origin of bursts and heavy tails in human dynamics. Nature 435, 7039 (2005), 207–211.
  3. Albert-László Barabási and Réka Albert. 1999. Emergence of Scaling in Random Networks. Science 286, 5439 (1999), 509. http://search.ebscohost.com/login.aspx?direct=true&db=tfh&AN=2405932&site=ehost-live
  4. Language models are few-shot learners. Advances in neural information processing systems 33 (2020), 1877–1901.
  5. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018).
  6. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. https://arxiv.org/abs/1810.04805
  7. Zero-shot recommender systems. arXiv preprint arXiv:2105.08318 (2021).
  8. Mamo: Memory-augmented meta-optimization for cold-start recommendation. In Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining. 688–697.
  9. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020).
  10. How to learn item representation for cold-start multimedia recommendation?. In Proceedings of the 28th ACM International Conference on Multimedia. 3469–3477.
  11. Zero shot on the cold-start problem: Model-agnostic interest learning for recommender systems. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management. 474–483.
  12. Pre-training graph neural networks for cold-start users and items representation. In Proceedings of the 14th ACM International Conference on Web Search and Data Mining. 265–273.
  13. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770–778.
  14. LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation. CoRR abs/2002.02126 (2020). arXiv:2002.02126 https://arxiv.org/abs/2002.02126
  15. Neural collaborative filtering. In WWW. 173–182.
  16. Session-based recommendations with recurrent neural networks. arXiv preprint arXiv:1511.06939 (2015).
  17. Parallel recurrent neural network architectures for feature-rich session-based recommendations. In Proceedings of the 10th ACM conference on recommender systems. 241–248.
  18. Towards universal sequence representation learning for recommender systems. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 585–593.
  19. Conet: Collaborative cross networks for cross-domain recommendation. In Proceedings of the 27th ACM international conference on information and knowledge management. 667–676.
  20. A re-visit of the popularity baseline in recommender systems. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 1749–1752.
  21. Wang-Cheng Kang and Julian McAuley. 2018. Self-attentive sequential recommendation. In 2018 IEEE international conference on data mining (ICDM). IEEE, 197–206.
  22. Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).
  23. Thomas N Kipf and Max Welling. 2016. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016).
  24. Yehuda Koren. 2008. Factorization meets the neighborhood: a multifaceted collaborative filtering model. In Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining. 426–434.
  25. Yehuda Koren. 2009. Collaborative filtering with temporal dynamics. In Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining. 447–456.
  26. MeLU: Meta-Learned User Preference Estimator for Cold-Start Recommendation. In KDD. 1073–1082.
  27. Neural attentive session-based recommendation. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management. 1419–1428.
  28. Time interval aware self-attention for sequential recommendation. In Proceedings of the 13th international conference on web search and data mining. 322–330.
  29. Pan Li and Alexander Tuzhilin. 2020. Ddtcdr: Deep dual transfer cross domain recommendation. In Proceedings of the 13th International Conference on Web Search and Data Mining. 331–339.
  30. STAMP: short-term attention/memory priority model for session-based recommendation. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining. 1831–1839.
  31. Leveraging distribution alignment via stein path for cross-domain cold-start recommendation. Advances in Neural Information Processing Systems 34 (2021), 19223–19234.
  32. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692 (2019).
  33. Meta-learning on heterogeneous information networks for cold-start recommendation. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 1563–1573.
  34. SDM: Sequential deep matching model for online large-scale recommender system. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management. 2635–2643.
  35. Cross-domain recommendation: An embedding and mapping approach.. In IJCAI, Vol. 17. 2464–2470.
  36. Justifying recommendations using distantly-labeled reviews and fine-grained aspects. In Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP). 188–197.
  37. OpenAI. 2023. GPT-4 Technical Report. arXiv:cs.CL/2303.08774
  38. Learning transferable visual models from natural language supervision. In International conference on machine learning. PMLR, 8748–8763.
  39. Exploring the limits of transfer learning with a unified text-to-text transformer. The Journal of Machine Learning Research 21, 1 (2020), 5485–5551.
  40. BPR: Bayesian personalized ranking from implicit feedback. In UAI. AUAI Press, 452–461.
  41. Factorizing personalized markov chains for next-basket recommendation. In Proceedings of the 19th international conference on World wide web. 811–820.
  42. Experimental Study of Inequality and Unpredictability in an Artificial Cultural Market. Science 311, 5762 (2006), 854–856. http://www.jstor.org/stable/3843620
  43. An MDP-based recommender system. Journal of Machine Learning Research 6, 9 (2005).
  44. Session-based social recommendation via dynamic graph attention networks. In Proceedings of the Twelfth ACM international conference on web search and data mining. 555–563.
  45. BERT4Rec: Sequential recommendation with bidirectional encoder representations from transformer. In Proceedings of the 28th ACM international conference on information and knowledge management. 1441–1450.
  46. Dynamic memory based attention network for sequential recommendation. In Proceedings of the AAAI conference on artificial intelligence, Vol. 35. 4384–4392.
  47. Improved recurrent neural networks for session-based recommendations. In Proceedings of the 1st workshop on deep learning for recommender systems. 17–22.
  48. eTrust: Understanding trust evolution in an online world. , 253–261 pages.
  49. mTrust: Discerning multi-faceted trust in a connected world. In Proceedings of the fifth ACM international conference on Web search and data mining. ACM, 93–102.
  50. Jiaxi Tang and Ke Wang. 2018. Personalized top-n sequential recommendation via convolutional sequence embedding. In Proceedings of the eleventh ACM international conference on web search and data mining. 565–573.
  51. Trinh Xuan Tuan and Tu Minh Phuong. 2017. 3D convolutional networks for session-based recommendation with content features. In Proceedings of the eleventh ACM conference on recommender systems. 138–146.
  52. Attention is all you need. Advances in neural information processing systems 30 (2017).
  53. Dropoutnet: Addressing cold start in recommender systems. In Advances in neural information processing systems. 4957–4966.
  54. Pre-trained Neural Recommenders: A Transferable Zero-Shot Framework for Recommendation Systems. arXiv:cs.IR/2309.01188
  55. Neural graph collaborative filtering. In SIGIR. 165–174.
  56. Contrastive learning for cold-start recommendation. In Proceedings of the 29th ACM International Conference on Multimedia. 5382–5390.
  57. Recurrent recommender networks. In Proceedings of the tenth ACM international conference on web search and data mining. 495–503.
  58. Sequential recommender system based on hierarchical attention network. In IJCAI International Joint Conference on Artificial Intelligence.
  59. Multi-order attentive ranking model for sequential recommendation. In Proceedings of the AAAI conference on artificial intelligence, Vol. 33. 5709–5716.
  60. Parameter-Efficient Transfer from Sequential Behaviors for User Modeling and Recommendation. Proceedings of the 42nd international ACM SIGIR conference on Research and development in Information Retrieval (2020).
  61. Latent factor transition for dynamic collaborative filtering. In Proceedings of the 2014 SIAM international conference on data mining. SIAM, 452–460.
  62. CATN: Cross-domain recommendation for cold-start users via aspect transfer network. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 229–238.
  63. Transfer-meta framework for cross-domain recommendation to cold-start users. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. 1813–1817.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (3)
  1. Junting Wang (8 papers)
  2. Praneet Rathi (5 papers)
  3. Hari Sundaram (46 papers)

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets