Papers
Topics
Authors
Recent
Search
2000 character limit reached

End-to-end Learnable Clustering for Intent Learning in Recommendation

Published 11 Jan 2024 in cs.IR and cs.AI | (2401.05975v5)

Abstract: Intent learning, which aims to learn users' intents for user understanding and item recommendation, has become a hot research spot in recent years. However, existing methods suffer from complex and cumbersome alternating optimization, limiting performance and scalability. To this end, we propose a novel intent learning method termed \underline{ELCRec}, by unifying behavior representation learning into an \underline{E}nd-to-end \underline{L}earnable \underline{C}lustering framework, for effective and efficient \underline{Rec}ommendation. Concretely, we encode user behavior sequences and initialize the cluster centers (latent intents) as learnable neurons. Then, we design a novel learnable clustering module to separate different cluster centers, thus decoupling users' complex intents. Meanwhile, it guides the network to learn intents from behaviors by forcing behavior embeddings close to cluster centers. This allows simultaneous optimization of recommendation and clustering via mini-batch data. Moreover, we propose intent-assisted contrastive learning by using cluster centers as self-supervision signals, further enhancing mutual promotion. Both experimental results and theoretical analyses demonstrate the superiority of ELCRec from six perspectives. Compared to the runner-up, ELCRec improves NDCG@5 by 8.9\% and reduces computational costs by 22.5\% on the Beauty dataset. Furthermore, due to the scalability and universal applicability, we deploy this method on the industrial recommendation system with 130 million page views and achieve promising results. The codes are available on GitHub (https://github.com/yueliu1999/ELCRec). A collection (papers, codes, datasets) of deep group recommendation/intent learning methods is available on GitHub (https://github.com/yueliu1999/Awesome-Deep-Group-Recommendation).

Definition Search Book Streamline Icon: https://streamlinehq.com
References (84)
  1. {{\{{TensorFlow}}\}}: a system for {{\{{Large-Scale}}\}} machine learning. In 12th USENIX symposium on operating systems design and implementation (OSDI 16) (2016), pp. 265–283.
  2. A survey of text clustering algorithms. Mining text data (2012), 77–128.
  3. Self-labelling via simultaneous clustering and representation learning. In International Conference on Learning Representations (2019).
  4. Rademacher and gaussian complexities: Risk bounds and structural results. Journal of Machine Learning Research 3, Nov (2002), 463–482.
  5. Deep clustering for unsupervised learning of visual features. In Proc. of ECCV (2018).
  6. Unsupervised learning of visual features by contrasting cluster assignments. Advances in neural information processing systems 33 (2020), 9912–9924.
  7. Emerging properties in self-supervised vision transformers. In Proceedings of the IEEE/CVF international conference on computer vision (2021), pp. 9650–9660.
  8. Controllable multi-interest framework for recommendation. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (2020), pp. 2942–2951.
  9. Latent user intent modeling for sequential recommenders. In Companion Proceedings of the ACM Web Conference 2023 (2023), pp. 427–431.
  10. Sequential recommendation with graph neural networks. In Proceedings of the 44th international ACM SIGIR conference on research and development in information retrieval (2021), pp. 378–387.
  11. Deep adaptive image clustering. In Proceedings of the IEEE international conference on computer vision (2017), pp. 5879–5887.
  12. Intent contrastive learning for sequential recommendation. In Proceedings of the ACM Web Conference 2022 (2022), pp. 2172–2182.
  13. Mean shift: A robust approach toward feature space analysis. IEEE Transactions on pattern analysis and machine intelligence 24, 5 (2002), 603–619.
  14. A density-based algorithm for discovering clusters in large spatial databases with noise. In kdd (1996), vol. 96, pp. 226–231.
  15. Neighborhood-based hard negative mining for sequential recommendation. arXiv preprint arXiv:2306.10047 (2023).
  16. Sequential recommendation via stochastic self-attention. In Proceedings of the ACM Web Conference 2022 (2022), pp. 2036–2047.
  17. Continuous-time sequential recommendation with temporal graph collaborative transformer. In Proceedings of the 30th ACM international conference on information & knowledge management (2021), pp. 433–442.
  18. Improved deep embedded clustering with local structure preservation. In Proc. of IJCAI (2017).
  19. Algorithm as 136: A k-means clustering algorithm. Journal of the royal statistical society. series c (applied statistics) (1979).
  20. Fusing similarity models with markov chains for sparse sequential recommendation. In 2016 IEEE 16th international conference on data mining (ICDM) (2016), IEEE, pp. 191–200.
  21. Session-based recommendations with recurrent neural networks. arXiv preprint arXiv:1511.06939 (2015).
  22. Clustering documents based on semantic similarity using hac and k-mean algorithms. In 2020 International Conference on Advanced Science and Engineering (ICOASE) (2020), IEEE, pp. 205–210.
  23. Self-attentive sequential recommendation. In 2018 IEEE international conference on data mining (ICDM) (2018), IEEE, pp. 197–206.
  24. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114 (2013).
  25. Review on determining number of cluster in k-means clustering. International Journal 1, 6 (2013), 90–95.
  26. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems 25 (2012).
  27. Multi-interest network with dynamic routing for recommendation at tmall. In Proceedings of the 28th ACM international conference on information and knowledge management (2019), pp. 2615–2623.
  28. Intention-aware sequential recommendation with structured intent transition. IEEE Transactions on Knowledge and Data Engineering 34, 11 (2021), 5403–5414.
  29. Time interval aware self-attention for sequential recommendation. In Proceedings of the 13th international conference on web search and data mining (2020), pp. 322–330.
  30. Prototypical contrastive learning of unsupervised representations. In International Conference on Learning Representations (2020).
  31. Automlp: Automated mlp for sequential recommendations. In Proceedings of the ACM Web Conference 2023 (2023), pp. 1190–1198.
  32. Mlp4rec: A pure mlp architecture for sequential recommendations. arXiv preprint arXiv:2204.11510 (2022).
  33. Multi-intention oriented contrastive learning for sequential recommendation. In Proceedings of the Sixteenth ACM International Conference on Web Search and Data Mining (2023), pp. 411–419.
  34. Lightweight self-attentive sequential recommendation. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management (2021), pp. 967–977.
  35. Edge-enhanced global disentangled graph neural network for sequential recommendation. ACM Transactions on Knowledge Discovery from Data 17, 6 (2023), 1–22.
  36. Contrastive clustering. In Proceedings of the AAAI conference on artificial intelligence (2021), vol. 35, pp. 8547–8555.
  37. Open-world semantic segmentation via contrasting and clustering vision-language embedding. In European Conference on Computer Vision (2022), Springer, pp. 275–292.
  38. Deep graph clustering via dual correlation reduction. In Proceedings of the AAAI Conference on Artificial Intelligence (2022), vol. 36, pp. 7603–7611.
  39. Simple contrastive graph clustering. IEEE Transactions on Neural Networks and Learning Systems (2023).
  40. Hard sample aware network for contrastive deep graph clustering. In Proceedings of the AAAI conference on artificial intelligence (2023), vol. 37, pp. 8914–8922.
  41. Contrastive self-supervised sequential recommendation with robust augmentation. arXiv preprint arXiv:2108.06479 (2021).
  42. Augmenting sequential recommendation with pseudo-prior items via reversely pre-training transformer. In Proceedings of the 44th international ACM SIGIR conference on Research and development in information retrieval (2021), pp. 1608–1612.
  43. Basket recommendation with multi-intent translation graph neural network. In 2020 IEEE International Conference on Big Data (Big Data) (2020), IEEE, pp. 728–737.
  44. Modeling task relationships in multi-task learning with multi-gate mixture-of-experts. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining (2018), pp. 1930–1939.
  45. Disentangled self-supervision in sequential recommenders. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (2020), pp. 483–491.
  46. Image-based recommendations on styles and substitutes. In Proceedings of the 38th international ACM SIGIR conference on research and development in information retrieval (2015), pp. 43–52.
  47. A survey of clustering with deep learning: From the perspective of network architecture. IEEE Access (2018).
  48. Foundations of machine learning. MIT press, 2018.
  49. Nickerson, R. S. Confirmation bias: A ubiquitous phenomenon in many guises. Review of general psychology 2, 2 (1998), 175–220.
  50. Adversarially regularized graph autoencoder for graph embedding. arXiv preprint arXiv:1802.04407 (2018).
  51. An intent-guided collaborative machine for session-based recommendation. In Proceedings of the 43rd international ACM SIGIR conference on research and development in information retrieval (2020), pp. 1833–1836.
  52. gsasrec: Reducing overconfidence in sequential recommendation trained with negative sampling. In Proceedings of the 17th ACM Conference on Recommender Systems (2023), pp. 116–128.
  53. Qian, Q. Stable cluster discrimination for deep clustering. In Proceedings of the IEEE/CVF International Conference on Computer Vision (2023), pp. 16645–16654.
  54. Unsupervised visual representation learning by online constrained k-means. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2022), pp. 16640–16649.
  55. Rendle, S. Factorization machines. In 2010 IEEE International conference on data mining (2010), IEEE, pp. 995–1000.
  56. Bpr: Bayesian personalized ranking from implicit feedback. arXiv preprint arXiv:1205.2618 (2012).
  57. Factorizing personalized markov chains for next-basket recommendation. In Proceedings of the 19th international conference on World wide web (2010), pp. 811–820.
  58. Reynolds, D. A. Gaussian mixture models. Encyclopedia of biometrics (2009).
  59. Clustering by fast search and find of density peaks. science 344, 6191 (2014), 1492–1496.
  60. Deepdpm: Deep clustering with an unknown number of clusters. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2022), pp. 9861–9870.
  61. Dynamic routing between capsules. Advances in neural information processing systems 30 (2017).
  62. Unstructured text documents summarization with multi-stage clustering. IEEE Access 8 (2020), 212838–212854.
  63. Bert4rec: Sequential recommendation with bidirectional encoder representations from transformer. In Proceedings of the 28th ACM international conference on information and knowledge management (2019), pp. 1441–1450.
  64. Integration k-means clustering method and elbow method for identification of the best customer profile cluster. In IOP conference series: materials science and engineering (2018), vol. 336, IOP Publeishing, p. 012017.
  65. Personalized top-n sequential recommendation via convolutional sequence embedding. In Proceedings of the eleventh ACM international conference on web search and data mining (2018), pp. 565–573.
  66. Attentive sequential models of latent intent for next item recommendation. In Proceedings of The Web Conference 2020 (2020), pp. 2528–2534.
  67. Visualizing data using t-sne. Journal of machine learning research 9, 11 (2008).
  68. Attention is all you need. Advances in neural information processing systems 30 (2017).
  69. Von Luxburg, U. A tutorial on spectral clustering. Statistics and computing (2007).
  70. Attributed graph clustering: A deep attentional embedding approach. arXiv preprint arXiv:1906.06532 (2019).
  71. Modeling multi-purpose sessions for next-item recommendations via mixture-channel purpose routing networks. In International Joint Conference on Artificial Intelligence (2019), International Joint Conferences on Artificial Intelligence.
  72. Recurrent recommender networks. In Proceedings of the tenth ACM international conference on web search and data mining (2017), pp. 495–503.
  73. Unsupervised deep embedding for clustering analysis. In Proc. of ICML (2016).
  74. Contrastive learning for sequential recommendation. In 2022 IEEE 38th international conference on data engineering (ICDE) (2022), IEEE, pp. 1259–1273.
  75. Cosrec: 2d convolutional neural networks for sequential recommendation. In Proceedings of the 28th ACM international conference on information and knowledge management (2019), pp. 2173–2176.
  76. Joint unsupervised learning of deep representations and image clusters. In Proceedings of the IEEE conference on computer vision and pattern recognition (2016), pp. 5147–5156.
  77. Debiased contrastive learning for sequential recommendation. In Proceedings of the ACM Web Conference 2023 (2023), pp. 1063–1073.
  78. Graph masked autoencoder for sequential recommendation. arXiv preprint arXiv:2305.04619 (2023).
  79. Recurrent neural network regularization. arXiv preprint arXiv:1409.2329 (2014).
  80. Enhancing sequential recommendation with graph contrastive learning. arXiv preprint arXiv:2205.14837 (2022).
  81. Adaptive disentangled transformer for sequential recommendation. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (2023), pp. 3434–3445.
  82. S3-rec: Self-supervised learning for sequential recommendation with mutual information maximization. In Proceedings of the 29th ACM international conference on information & knowledge management (2020), pp. 1893–1902.
  83. Filter-enhanced mlp is all you need for sequential recommendation. In Proceedings of the ACM web conference 2022 (2022), pp. 2388–2399.
  84. Equivariant contrastive learning for sequential recommendation. In Proceedings of the 17th ACM Conference on Recommender Systems (2023), pp. 129–140.
Citations (4)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Summarize

Paper to Video (Beta)

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up

Tweets

Sign up for free to view the 1 tweet with 1 like about this paper.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up for Free