Papers
Topics
Authors
Recent
Search
2000 character limit reached

One Backpropagation in Two Tower Recommendation Models

Published 27 Mar 2024 in cs.IR | (2403.18227v4)

Abstract: Recent years have witnessed extensive researches on developing two tower recommendation models for relieving information overload. Four building modules can be identified in such models, namely, user-item encoding, negative sampling, loss computing and back-propagation updating. To the best of our knowledge, existing algorithms have researched only on the first three modules, yet neglecting the backpropagation module. They all adopt a kind of two backpropagation strategy, which are based on an implicit assumption of equally treating users and items in the training phase. In this paper, we challenge such an equal training assumption and propose a novel one backpropagation updating strategy, which keeps the normal gradient backpropagation for the item encoding tower, but cuts off the backpropagation for the user encoding tower. Instead, we propose a moving-aggregation updating strategy to update a user encoding in each training epoch. Except the proposed backpropagation updating module, we implement the other three modules with the most straightforward choices. Experiments on four public datasets validate the effectiveness and efficiency of our model in terms of improved recommendation performance and reduced computation overload over the state-of-the-art competitors.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (45)
  1. Information overload and usage of recommendations. In Proceedings of the ACM RecSys 2010 Workshop on User-Centric Evaluation of Recommender Systems and Their Interfaces (UCERSTI), Barcelona, Spain. Association for Computing Machinery, New York, NY, USA, 26–33.
  2. Using recommendation agents to cope with information overload. International Journal of Electronic Commerce 17, 2 (2012), 41–70.
  3. Rethinking collaborative metric learning: Toward an efficient alternative without negative sampling. IEEE Transactions on Pattern Analysis and Machine Intelligence 45, 1 (2022), 1017–1035.
  4. The minority matters: A diversity-promoting collaborative metric learning algorithm. Advances in Neural Information Processing Systems 35 (2022), 2451–2464.
  5. CFGAN: A Generic Collaborative Filtering Framework Based on Generative Adversarial Networks. In Proceedings of the 27th ACM International Conference on Information and Knowledge Management. Association for Computing Machinery, New York, NY, USA, 137–146.
  6. A simple framework for contrastive learning of visual representations. In International conference on machine learning. Association for Computing Machinery, New York, NY, USA, 1597–1607.
  7. On Sampling Strategies for Neural Network-based Collaborative Filtering. In KDD. Association for Computing Machinery, New York, NY, USA, 14.
  8. Incremental False Negative Detection for Contrastive Learning. In International Conference on Learning Representations. OpenReview.net, New York, NY, USA.
  9. Fairly adaptive negative sampling for recommendations. In Proceedings of the ACM Web Conference 2023. Association for Computing Machinery, New York, NY, USA, 3723–3733.
  10. Debiased contrastive learning. Advances in Neural Information Processing Systems 33 (2020), 8765–8775.
  11. Simplify and robustify negative sampling for implicit collaborative filtering. Advances in Neural Information Processing Systems 33 (2020), 1094–1105.
  12. Neural Collaborative Filtering. In WWW. International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE, 173–182.
  13. Collaborative Metric Learning. In WWW. International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE, 193–201.
  14. Collaborative filtering for implicit feedback datasets. In 2008 Eighth IEEE international conference on data mining. IEEE, Piscataway, 263–272.
  15. Learning deep structured semantic models for web search using clickthrough data. In Proceedings of the 22nd ACM International Conference on Information & Knowledge Management. Association for Computing Machinery, New York, NY, USA, 2333–2338.
  16. Supervised contrastive learning. Advances in neural information processing systems 33 (2020), 18661–18673.
  17. Matrix factorization techniques for recommender systems. Computer 42, 8 (2009), 30–37.
  18. Bootstrapping User and Item Representations for One-Class Collaborative Filtering. In SIGIR. Association for Computing Machinery, New York, NY, USA, 317–326.
  19. Your Negative May Not Be True Negative: Boosting Image-Text Matching with False Negative Elimination. In Proceedings of the 31st ACM International Conference on Multimedia. Association for Computing Machinery, New York, NY, USA, 924–934.
  20. FairGAN: GANs-Based Fairness-Aware Learning for Recommendations with Implicit Feedback. In Proceedings of the ACM Web Conference 2022. Association for Computing Machinery, New York, NY, USA, 297–307.
  21. Symmetric metric learning with adaptive margin for recommendation. Proceedings of the AAAI Conference on Artificial Intelligence 34, 04 (2020), 4634–4641.
  22. Variational Autoencoders for Collaborative Filtering. In Proceedings of the 2018 World Wide Web Conference. International World Wide Web Conferences Steering Committee, New York, NY, USA, 689–698.
  23. Reducing Popularity Bias in Recommender Systems through AUC-Optimal Negative Sampling. arXiv preprint arXiv:2306.01348 (2023).
  24. Bin Liu and Bang Wang. 2023. Bayesian Negative Sampling for Recommendation. In 2023 IEEE 39th International Conference on Data Engineering (ICDE). IEEE, Piscataway, 749–761.
  25. SimpleX: A Simple and Strong Baseline for Collaborative Filtering. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management. Association for Computing Machinery, New York, NY, USA, 1243–1252.
  26. Distributed Representations of Words and Phrases and Their Compositionality. Advances in neural information processing systems 26 (2013), 3111–3119.
  27. JERZY Neyman and ELIZABETH L SCOTT. 1967. Berkeley symposium on mathematical statistics and probability. In Proceedings of the Berkeley Symposium on Mathematical Statistics and Probability, eds LM Le Cam and J. Neyman (Berkeley, CA: University of California Press).
  28. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748 (2018).
  29. One-class collaborative filtering. In 2008 Eighth IEEE international conference on data mining. IEEE, Piscataway, 502–511.
  30. Relation-aware graph attention model with adaptive self-adversarial training. Proceedings of the AAAI Conference on Artificial Intelligence 35, 11 (2021), 9368–9376.
  31. BPR: Bayesian Personalized Ranking from Implicit Feedback. In UAI 2009, Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence, Montreal, QC, Canada, June 18-21, 2009. AUAI Press, Arlington, Virginia, USA, 452–461.
  32. Contrastive Learning with Hard Negative Samples. In International Conference on Learning Representations. OpenReview.net, New York, NY, USA, 2801.
  33. Detection of False Positive and False Negative Samples in Semantic Segmentation. In Proceedings of the 23rd Conference on Design, Automation and Test in Europe. EDA Consortium, San Jose, CA, USA, 1351–1356.
  34. Jiaming Song and Stefano Ermon. 2020. Multi-label contrastive predictive coding. Advances in neural information processing systems 33 (2020), 8161–8173.
  35. Beyond Two-Tower Matching: Learning Sparse Retrievable Cross-Interactions for Recommendation. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval. Association for Computing Machinery, New York, NY, USA, 548–557.
  36. LINE: Large-Scale Information Network Embedding. In Proceedings of the 24th International Conference on World Wide Web. International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE, 1067–1077.
  37. Improving Collaborative Metric Learning with Efficient Negative Sampling. In SIGIR. Association for Computing Machinery, New York, NY, USA, 1201–1204.
  38. Laurens van der Maaten and Geoffrey E. Hinton. 2008. Visualizing Data using t-SNE. Journal of Machine Learning Research 9 (2008), 2579–2605.
  39. Neural Graph Collaborative Filtering. In SIGIR. Association for Computing Machinery, New York, NY, USA, 165–174.
  40. Reinforced Negative Sampling over Knowledge Graph for Recommendation. In Proceedings of The Web Conference 2020. Association for Computing Machinery, New York, NY, USA, 99–109.
  41. Collaborative Residual Metric Learning. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval. Association for Computing Machinery, New York, NY, USA, 1107–1116.
  42. LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation.. In SIGIR. Association for Computing Machinery, New York, NY, USA, 10.
  43. Mixed Negative Sampling for Learning Two-tower Neural Networks in Recommendations. In Companion Proceedings of the Web Conference 2020. Association for Computing Machinery, New York, NY, USA, 441–447.
  44. Multiple pairwise ranking with implicit feedback. In Proceedings of the 27th ACM Int. Conference on Information and Knowledge Management, ACM. Association for Computing Machinery, New York, NY, USA, 1727–1730.
  45. Optimizing Top-n Collaborative Filtering via Dynamic Negative Item Sampling. In Proceedings of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval. Association for Computing Machinery, New York, NY, USA, 785–788.

Summary

No one has generated a summary of this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Summarize

Paper to Video (Beta)

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Authors (2)

  1. Erjia Chen 
  2. Bang Wang 

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up