Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Learning Graph ODE for Continuous-Time Sequential Recommendation (2304.07042v2)

Published 14 Apr 2023 in cs.IR

Abstract: Sequential recommendation aims at understanding user preference by capturing successive behavior correlations, which are usually represented as the item purchasing sequences based on their past interactions. Existing efforts generally predict the next item via modeling the sequential patterns. Despite effectiveness, there exist two natural deficiencies: (i) user preference is dynamic in nature, and the evolution of collaborative signals is often ignored; and (ii) the observed interactions are often irregularly-sampled, while existing methods model item transitions assuming uniform intervals. Thus, how to effectively model and predict the underlying dynamics for user preference becomes a critical research problem. To tackle the above challenges, in this paper, we focus on continuous-time sequential recommendation and propose a principled graph ordinary differential equation framework named GDERec. Technically, GDERec is characterized by an autoregressive graph ordinary differential equation consisting of two components, which are parameterized by two tailored graph neural networks (GNNs) respectively to capture user preference from the perspective of hybrid dynamical systems. The two customized GNNs are trained alternately in an autoregressive manner to track the evolution of the underlying system from irregular observations, and thus learn effective representations of users and items beneficial to the sequential recommendation. Extensive experiments on five benchmark datasets demonstrate the superiority of our model over various state-of-the-art recommendation methods.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (66)
  1. J. Wang, P. Huang, H. Zhao, Z. Zhang, B. Zhao, and D. L. Lee, “Billion-scale commodity embedding for e-commerce recommendation in alibaba,” in Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2018, pp. 839–848.
  2. Z. Li, X. Shen, Y. Jiao, X. Pan, P. Zou, X. Meng, C. Yao, and J. Bu, “Hierarchical bipartite graph neural networks: Towards large-scale e-commerce applications,” in 2020 IEEE 36th International Conference on Data Engineering (ICDE).   IEEE, 2020, pp. 1677–1688.
  3. X. Zhang, C. Zhang, X. Li, X. L. Dong, J. Shang, C. Faloutsos, and J. Han, “Oa-mine: Open-world attribute mining for e-commerce products with weak supervision,” in Proceedings of the ACM Web Conference 2022, 2022, pp. 3153–3161.
  4. E. Min, Y. Rong, Y. Bian, T. Xu, P. Zhao, J. Huang, and S. Ananiadou, “Divide-and-conquer: Post-user interaction network for fake news detection on social media,” in Proceedings of the ACM Web Conference 2022, 2022, pp. 1148–1158.
  5. M. Mousavi, H. Davulcu, M. Ahmadi, R. Axelrod, R. Davis, and S. Atran, “Effective messaging on social media: What makes online content go viral?” in Proceedings of the ACM Web Conference 2022, 2022, pp. 2957–2966.
  6. J. Song, K. Han, and S.-W. Kim, ““i have no text in my post”: Using visual hints to model user emotions in social media,” in Proceedings of the ACM Web Conference 2022, 2022, pp. 2888–2896.
  7. A. Mnih and R. R. Salakhutdinov, “Probabilistic matrix factorization,” Advances in neural information processing systems, vol. 20, 2007.
  8. P. Gopalan, J. M. Hofman, and D. M. Blei, “Scalable recommendation with hierarchical poisson factorization.” in UAI, 2015, pp. 326–335.
  9. Y. Koren, R. Bell, and C. Volinsky, “Matrix factorization techniques for recommender systems,” Computer, vol. 42, no. 8, pp. 30–37, 2009.
  10. S. Rendle, C. Freudenthaler, and L. Schmidt-Thieme, “Factorizing personalized markov chains for next-basket recommendation,” in Proceedings of the 19th international conference on World wide web, 2010, pp. 811–820.
  11. R. He and J. McAuley, “Fusing similarity models with markov chains for sparse sequential recommendation,” in 2016 IEEE 16th international conference on data mining (ICDM).   IEEE, 2016, pp. 191–200.
  12. S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural computation, vol. 9, no. 8, pp. 1735–1780, 1997.
  13. B. Hidasi, A. Karatzoglou, L. Baltrunas, and D. Tikk, “Session-based recommendations with recurrent neural networks,” arXiv preprint arXiv:1511.06939, 2015.
  14. M. Quadrana, A. Karatzoglou, B. Hidasi, and P. Cremonesi, “Personalizing session-based recommendations with hierarchical recurrent neural networks,” in proceedings of the Eleventh ACM Conference on Recommender Systems, 2017, pp. 130–137.
  15. Z. Li, H. Zhao, Q. Liu, Z. Huang, T. Mei, and E. Chen, “Learning from history and present: Next-item recommendation via discriminatively exploiting user behaviors,” in Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2018, pp. 1734–1743.
  16. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in neural information processing systems, vol. 30, 2017.
  17. J. Li, P. Ren, Z. Chen, Z. Ren, T. Lian, and J. Ma, “Neural attentive session-based recommendation,” in Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, 2017, pp. 1419–1428.
  18. W.-C. Kang and J. McAuley, “Self-attentive sequential recommendation,” in 2018 IEEE international conference on data mining (ICDM).   IEEE, 2018, pp. 197–206.
  19. F. Sun, J. Liu, J. Wu, C. Pei, X. Lin, W. Ou, and P. Jiang, “Bert4rec: Sequential recommendation with bidirectional encoder representations from transformer,” in Proceedings of the 28th ACM international conference on information and knowledge management, 2019, pp. 1441–1450.
  20. L. Wu, S. Li, C.-J. Hsieh, and J. Sharpnack, “Sse-pt: Sequential recommendation via personalized transformer,” in Fourteenth ACM Conference on Recommender Systems, 2020, pp. 328–337.
  21. Z. Fan, Z. Liu, J. Zhang, Y. Xiong, L. Zheng, and P. S. Yu, “Continuous-time sequential recommendation with temporal graph collaborative transformer,” in Proceedings of the 30th ACM International Conference on Information & Knowledge Management, 2021, pp. 433–442.
  22. Z. Fan, Z. Liu, Y. Wang, A. Wang, Z. Nazari, L. Zheng, H. Peng, and P. S. Yu, “Sequential recommendation via stochastic self-attention,” in Proceedings of the ACM Web Conference 2022, 2022, pp. 2036–2047.
  23. T. N. Kipf and M. Welling, “Semi-supervised classification with graph convolutional networks,” in Proceedings of International Conference on Learning Representations, 2017.
  24. X. Luo, W. Ju, M. Qu, Y. Gu, C. Chen, M. Deng, X.-S. Hua, and M. Zhang, “Clear: Cluster-enhanced contrast for self-supervised graph representation learning,” IEEE Transactions on Neural Networks and Learning Systems, 2022.
  25. W. Ju, Z. Fang, Y. Gu, Z. Liu, Q. Long, Z. Qiao, Y. Qin, J. Shen, F. Sun, Z. Xiao et al., “A comprehensive survey on deep graph representation learning,” arXiv preprint arXiv:2304.05055, 2023.
  26. W. Ju, X. Luo, M. Qu, Y. Wang, C. Chen, M. Deng, X.-S. Hua, and M. Zhang, “Tgnn: A joint semi-supervised framework for graph-level classification,” arXiv preprint arXiv:2304.11688, 2023.
  27. S. Wu, Y. Tang, Y. Zhu, L. Wang, X. Xie, and T. Tan, “Session-based recommendation with graph neural networks,” in Proceedings of the AAAI conference on artificial intelligence, vol. 33, no. 01, 2019, pp. 346–353.
  28. C. Xu, P. Zhao, Y. Liu, V. S. Sheng, J. Xu, F. Zhuang, J. Fang, and X. Zhou, “Graph contextualized self-attention network for session-based recommendation.” in IJCAI, vol. 19, 2019, pp. 3940–3946.
  29. Y. Yang, C. Huang, L. Xia, Y. Liang, Y. Yu, and C. Li, “Multi-behavior hypergraph-enhanced transformer for sequential recommendation,” in Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2022, pp. 2263–2274.
  30. Y. Qin, Y. Wang, F. Sun, W. Ju, X. Hou, Z. Wang, J. Cheng, J. Lei, and M. Zhang, “Disenpoi: Disentangling sequential and geographical influence for point-of-interest recommendation,” in Proceedings of the Sixteenth ACM International Conference on Web Search and Data Mining, 2023, pp. 508–516.
  31. X. Luo, W. Ju, M. Qu, C. Chen, M. Deng, X.-S. Hua, and M. Zhang, “Dualgraph: Improving semi-supervised graph classification via dual contrastive learning,” in 2022 IEEE 38th International Conference on Data Engineering (ICDE).   IEEE, 2022, pp. 699–712.
  32. W. Ju, Y. Gu, B. Chen, G. Sun, Y. Qin, X. Liu, X. Luo, and M. Zhang, “Glcc: A general framework for graph-level clustering,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 37, no. 4, 2023, pp. 4391–4399.
  33. W. Ju, Y. Gu, X. Luo, Y. Wang, H. Yuan, H. Zhong, and M. Zhang, “Unsupervised graph-level representation learning with hierarchical contrasts,” Neural Networks, vol. 158, pp. 359–368, 2023.
  34. J. Gilmer, S. S. Schoenholz, P. F. Riley, O. Vinyals, and G. E. Dahl, “Neural message passing for quantum chemistry,” in Proceedings of International Conference on Machine Learning, 2017, pp. 1263–1272.
  35. W. Song, Z. Xiao, Y. Wang, L. Charlin, M. Zhang, and J. Tang, “Session-based social recommendation via dynamic graph attention networks,” in Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining, 2019, pp. 555–563.
  36. Y. Wang, Y. Qin, F. Sun, B. Zhang, X. Hou, K. Hu, J. Cheng, J. Lei, and M. Zhang, “Disenctr: Dynamic graph-based disentangled representation for click-through rate prediction,” in Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2022, pp. 2314–2318.
  37. W. Ju, Y. Qin, Z. Qiao, X. Luo, Y. Wang, Y. Fu, and M. Zhang, “Kernel-based substructure exploration for next poi recommendation,” in 2022 IEEE International Conference on Data Mining (ICDM).   IEEE, 2022, pp. 221–230.
  38. Y. Qin, H. Wu, W. Ju, X. Luo, and M. Zhang, “A diffusion model for poi recommendation,” arXiv preprint arXiv:2304.07041, 2023.
  39. P. Veličković, G. Cucurull, A. Casanova, A. Romero, P. Lio, and Y. Bengio, “Graph attention networks,” in Proceedings of International Conference on Learning Representations, 2017.
  40. R. T. Chen, Y. Rubanova, J. Bettencourt, and D. K. Duvenaud, “Neural ordinary differential equations,” Advances in neural information processing systems, vol. 31, 2018.
  41. K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778.
  42. Z. Fang, Q. Long, G. Song, and K. Xie, “Spatial-temporal graph ode networks for traffic flow forecasting,” in Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, 2021, pp. 364–373.
  43. J. Choi, H. Choi, J. Hwang, and N. Park, “Graph neural controlled differential equations for traffic forecasting,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, no. 6, 2022, pp. 6367–6374.
  44. J. Ji, J. Wang, Z. Jiang, J. Jiang, and H. Zhang, “Stden: Towards physics-guided neural networks for traffic flow prediction,” 2022.
  45. X. Rao, H. Wang, L. Zhang, J. Li, S. Shang, and P. Han, “Fogs: First-order gradient supervision with learning-based graph for traffic flow forecasting,” in Proceedings of International Joint Conference on Artificial Intelligence, IJCAI.   ijcai. org, 2022.
  46. Y. Chen, B. Yang, Q. Meng, Y. Zhao, and A. Abraham, “Time-series forecasting using a system of ordinary differential equations,” Information Sciences, vol. 181, no. 1, pp. 106–114, 2011.
  47. E. De Brouwer, J. Simm, A. Arany, and Y. Moreau, “Gru-ode-bayes: Continuous modeling of sporadically-observed time series,” Advances in neural information processing systems, vol. 32, 2019.
  48. M. Jin, Y. Zheng, Y.-F. Li, S. Chen, B. Yang, and S. Pan, “Multivariate time series forecasting with dynamic graph neural odes,” arXiv preprint arXiv:2202.08408, 2022.
  49. Z. Huang, Y. Sun, and W. Wang, “Learning continuous system dynamics from irregularly-sampled partial observations,” Advances in Neural Information Processing Systems, vol. 33, pp. 16 177–16 187, 2020.
  50. ——, “Coupled graph ode for learning interacting system dynamics.” in KDD, 2021, pp. 705–715.
  51. X. Luo, J. Yuan, Z. Huang, H. Jiang, Y. Qin, W. Ju, M. Zhang, and Y. Sun, “Hope: High-order graph ode for modeling interacting dynamics,” 2023.
  52. J. Zhuang, N. Dvornek, X. Li, and J. S. Duncan, “Ordinary differential equations on graph networks,” 2019.
  53. L.-P. Xhonneux, M. Qu, and J. Tang, “Continuous graph neural networks,” in International Conference on Machine Learning.   PMLR, 2020, pp. 10 432–10 441.
  54. X. Wang, X. He, M. Wang, F. Feng, and T.-S. Chua, “Neural graph collaborative filtering,” in Proceedings of the 42nd international ACM SIGIR conference on Research and development in Information Retrieval, 2019, pp. 165–174.
  55. C. Runge, “Über die numerische auflösung von differentialgleichungen,” Mathematische Annalen, vol. 46, no. 2, pp. 167–178, 1895.
  56. D. Xu, C. Ruan, E. Korpeoglu, S. Kumar, and K. Achan, “Self-attention with functional time representation learning,” Advances in neural information processing systems, vol. 32, 2019.
  57. S. Rendle, C. Freudenthaler, Z. Gantner, and L. Schmidt-Thieme, “Bpr: Bayesian personalized ranking from implicit feedback,” arXiv preprint arXiv:1205.2618, 2012.
  58. J. R. Dormand and P. J. Prince, “A family of embedded runge-kutta formulae,” Journal of computational and applied mathematics, vol. 6, no. 1, pp. 19–26, 1980.
  59. X. He, K. Deng, X. Wang, Y. Li, Y. Zhang, and M. Wang, “Lightgcn: Simplifying and powering graph convolution network for recommendation,” in Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval, 2020, pp. 639–648.
  60. J. Guo, P. Zhang, C. Li, X. Xie, Y. Zhang, and S. Kim, “Evolutionary preference learning via graph nested gru ode for session-based recommendation,” in Proceedings of the 31st ACM International Conference on Information & Knowledge Management, 2022, pp. 624–634.
  61. J. Li, Y. Wang, and J. McAuley, “Time interval aware self-attention for sequential recommendation,” in Proceedings of the 13th international conference on web search and data mining, 2020, pp. 322–330.
  62. X. Wang, H. Jin, A. Zhang, X. He, T. Xu, and T.-S. Chua, “Disentangled graph collaborative filtering,” in Proceedings of the 43rd international ACM SIGIR conference on research and development in information retrieval, 2020, pp. 1001–1010.
  63. J. Bao and Y. Zhang, “Time-aware recommender system via continuous-time modeling,” in Proceedings of the 30th ACM International Conference on Information & Knowledge Management, 2021, pp. 2872–2876.
  64. X. Li, A. Sun, M. Zhao, J. Yu, K. Zhu, D. Jin, M. Yu, and R. Yu, “Multi-intention oriented contrastive learning for sequential recommendation,” in Proceedings of the Sixteenth ACM International Conference on Web Search and Data Mining, 2023, pp. 411–419.
  65. Y. Yang, C. Huang, L. Xia, C. Huang, D. Luo, and K. Lin, “Debiased contrastive learning for sequential recommendation,” in Proceedings of the ACM Web Conference 2023, 2023, pp. 1063–1073.
  66. L. Van der Maaten and G. Hinton, “Visualizing data using t-sne.” Journal of machine learning research, vol. 9, no. 11, 2008.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (5)
  1. Yifang Qin (22 papers)
  2. Wei Ju (46 papers)
  3. Hongjun Wu (12 papers)
  4. Xiao Luo (112 papers)
  5. Ming Zhang (313 papers)
Citations (19)

Summary

We haven't generated a summary for this paper yet.