Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
102 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

RDGCL: Reaction-Diffusion Graph Contrastive Learning for Recommendation (2312.16563v2)

Published 27 Dec 2023 in cs.IR, cs.AI, and cs.LG

Abstract: Contrastive learning (CL) has emerged as a promising technique for improving recommender systems, addressing the challenge of data sparsity by using self-supervised signals from raw data. Integration of CL with graph convolutional network (GCN)-based collaborative filterings (CFs) has been explored in recommender systems. However, current CL-based recommendation models heavily rely on low-pass filters and graph augmentations. In this paper, inspired by the reaction-diffusion equation, we propose a novel CL method for recommender systems called the reaction-diffusion graph contrastive learning model (RDGCL). We design our own GCN for CF based on the equations of diffusion, i.e., low-pass filter, and reaction, i.e., high-pass filter. Our proposed CL-based training occurs between reaction and diffusion-based embeddings, so there is no need for graph augmentations. Experimental evaluation on 5 benchmark datasets demonstrates that our proposed method outperforms state-of-the-art CL-based recommendation models. By enhancing recommendation accuracy and diversity, our method brings an advancement in CL for recommender systems.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (80)
  1. Samuel M Allen and John W Cahn. 1979. A microscopic theory for antiphase boundary motion and its application to antiphase domain coarsening. Acta metallurgica 27, 6 (1979), 1085–1095.
  2. A survey on data augmentation for text classification. Comput. Surveys 55, 7 (2022), 1–39.
  3. LightGCL: Simple Yet Effective Graph Contrastive Learning for Recommendation. In ICLR.
  4. GRAND: Graph Neural Diffusion. In ICML.
  5. Measuring and relieving the over-smoothing problem for graph neural networks from the topological view. In AAAI, Vol. 34. 3438–3445.
  6. Revisiting Graph Based Collaborative Filtering: A Linear Residual Graph Convolutional Network Approach. In AAAI.
  7. Neural Ordinary Differential Equations. In NeurIPS.
  8. A simple framework for contrastive learning of visual representations. In ICML. PMLR, 1597–1607.
  9. Big Self-Supervised Models are Strong Semi-Supervised Learners. arXiv preprint arXiv:2006.10029 (2020).
  10. Intriguing properties of contrastive losses. NeruIPS 34 (2021), 11834–11845.
  11. Yunjin Chen and Thomas Pock. 2016. Trainable nonlinear reaction diffusion: A flexible framework for fast and effective image restoration. IEEE transactions on pattern analysis and machine intelligence 39, 6 (2016), 1256–1272.
  12. On learning optimized reaction diffusion processes for effective image restoration. In CVPR. 5261–5269.
  13. Climate modeling with neural advection–diffusion equation. Knowledge and Information Systems 65, 6 (2023), 2403–2427.
  14. Blurring-Sharpening Process Models for Collaborative Filtering. In SIGIR.
  15. GREAD: Graph Neural Reaction-Diffusion Networks. In ICML.
  16. LT-OCF: Learnable-Time ODE-based Collaborative Filtering. In CIKM.
  17. Graph Convolutions Enrich the Self-Attention in Transformers! arXiv preprint arXiv:2312.04234 (2023).
  18. DiffCSE: Difference-based Contrastive Learning for Sentence Embeddings. In NAACL.
  19. Fan RK Chung. 1997. Spectral graph theory. Vol. 92. American Mathematical Soc.
  20. G-H Cottet and L Germain. 1993. Image processing through reaction combined with nonlinear diffusion. mathematics of computation (1993), 659–673.
  21. Graph neural networks as gradient flows. arXiv preprint arXiv:2206.10991 (2022).
  22. Venkatesan N. Ekambaram. 2013. Graph Structured Data Viewed Through a Fourier Lens. University of California, Berkeley (2013).
  23. Julio Esclarín and Luis Alvarez. 1997. Image quantization using reaction-diffusion equations. SIAM J. Appl. Math. 57, 1 (1997), 153–175.
  24. Graph Trend Filtering Networks for Recommendation. In SIGIR. 112–121.
  25. Ronald Aylmer Fisher. 1937. The wave of advance of advantageous genes. Annals of eugenics 7, 4 (1937), 355–369.
  26. William L. Hamilton. 2020. Graph Representation Learning. Synthesis Lectures on Artificial Intelligence and Machine Learning 14, 3 (2020), 1–159.
  27. F Maxwell Harper and Joseph A Konstan. 2015. The movielens datasets: History and context. Acm transactions on interactive intelligent systems (tiis) 5, 4 (2015), 1–19.
  28. Kaveh Hassani and Amir Hosein Khasahmadi. 2020. Contrastive multi-view representation learning on graphs. In ICML. PMLR, 4116–4126.
  29. Momentum contrast for unsupervised visual representation learning. In CVPR. 9729–9738.
  30. LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation. In SIGIR.
  31. Evaluating Collaborative Filtering Recommender Systems. ACM Trans. Inf. Syst. 22, 1 (2004), 5–53.
  32. TimeKit: A Time-series Forecasting-based Upgrade Kit for Collaborative Filtering. In 2022 IEEE International Conference on Big Data (Big Data). IEEE, 565–574.
  33. MGDCF: Distance Learning via Markov Graph Diffusion for Neural Collaborative Filtering. arXiv preprint arXiv: Arxiv-2204.02338 (2022).
  34. Climate modeling with neural diffusion equations. In ICDM. IEEE, 230–239.
  35. A survey on contrastive self-supervised learning. Technologies 9, 1 (2020), 2.
  36. Contrastive Self-supervised Learning in Recommender Systems: A Survey. arXiv preprint arXiv: Arxiv-2303.09902 (2023).
  37. Supervised contrastive learning. NeurIPS 33 (2020), 18661–18673.
  38. Shigeru Kondo and Takashi Miura. 2010. Reaction-diffusion model as a framework for understanding biological pattern formation. science 329, 5999 (2010), 1616–1620.
  39. Linear, or Non-Linear, That is the Question!. In WSDM. 517–525.
  40. gOCCF: Graph-theoretic one-class collaborative filtering based on uninteresting items. In AAAI, Vol. 32.
  41. SGCCL: siamese graph contrastive consensus learning for personalized recommendation. In WSDM. 589–597.
  42. Yongkun Li and Yang Kuang. 2001. Periodic solutions of periodic delay Lotka–Volterra equations and systems. J. Math. Anal. Appl. 255, 1 (2001), 260–280.
  43. Interest-Aware Message-Passing GCN for Recommendation. In TheWebConf (former WWW). 1296–1305.
  44. Self-supervised learning: Generative or contrastive. IEEE Transactions on Knowledge and Data Engineering 35, 1 (2021), 857–876.
  45. Alfred J Lotka. 2002. Contribution to the theory of periodic reactions. The Journal of Physical Chemistry 14, 3 (2002), 271–274.
  46. SimpleX: A Simple and Strong Baseline for Collaborative Filtering. In CIKM. 1243–1252.
  47. UltraGCN: Ultra Simplification of Graph Convolutional Networks for Recommendation. In CIKM.
  48. Edge detection with reaction-diffusion equations having a local average threshold. Pattern Recognition and Image Analysis 18 (2008), 289–299.
  49. Hoang Nt and Takanori Maehara. 2019. Revisiting graph neural networks: All we have is low-pass filters. arXiv preprint arXiv:1905.09550 (2019).
  50. Kenta Oono and Taiji Suzuki. 2020. Graph neural networks exponentially lose expressive power for node classification. In ICLR.
  51. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748 (2018).
  52. Less is More: Reweighting Important Spectral Graph Features for Recommendation. In SIGIR. 1273–1282.
  53. Zehra Pinar. 2021. An Analytical Studies of the Reaction-Diffusion Systems of Chemical Reactions. International Journal of Applied and Computational Mathematics 7, 3 (2021), 81.
  54. Gerlind Plonka and Jianwei Ma. 2008. Nonlinear regularized reaction-diffusion filters for denoising of images with textures. IEEE Transactions on Image Processing 17, 8 (2008), 1283–1294.
  55. GCC: Graph contrastive coding for graph neural network pre-training. In KDD. 1150–1160.
  56. Learning transferable visual models from natural language supervision. In ICML. PMLR, 8748–8763.
  57. A Survey on Oversmoothing in Graph Neural Networks. arXiv preprint arXiv: Arxiv-2303.10993 (2023).
  58. Uwe Schmidt and Stefan Roth. 2014. Shrinkage fields for effective image restoration. In CVPR. 2774–2781.
  59. How Powerful is Graph Convolution for Recommendation?. In CIKM.
  60. An Attentive Inductive Bias for Sequential Recommendation Beyond the Self-Attention. arXiv preprint arXiv:2312.10325 (2023).
  61. Alan Turing. 1952. The chemical basis of morphogenesis. Phil. Trans. R. Soc. Lond. B (1952).
  62. Greg Turk. 1991. Generating textures on arbitrary surfaces using reaction-diffusion. Acm Siggraph Computer Graphics 25, 4 (1991), 289–298.
  63. Greg Turk. 1992. Texturing surfaces using reaction-diffusion. The University of North Carolina at Chapel Hill.
  64. Neural Graph Collaborative Filtering. In SIGIR.
  65. Dissecting the Diffusion Process in Linear Graph Convolutional Networks. In NeurIPS.
  66. ACMP: Allen-Cahn Message Passing for Graph Neural Networks with Particle Phase Transition. In ICLR.
  67. Andrew Werth. 2015. Turing patterns in Photoshop. In Proceedings of Bridges 2015: Mathematics, Music, Art, Architecture, Culture. 459–462.
  68. Andrew Witkin and Michael Kass. 1991. Reaction-diffusion textures. In Proceedings of the 18th annual conference on computer graphics and interactive techniques. 299–308.
  69. Self-Supervised Graph Learning for Recommendation. In SIGIR. 726–735.
  70. SimGRACE: A Simple Framework for Graph Contrastive Learning without Data Augmentation. In TheWebConf (former WWW).
  71. Hypergraph contrastive collaborative filtering. In SIGIR. 70–79.
  72. Self-supervised hypergraph transformer for recommender systems. In KDD. 2100–2109.
  73. SimDCL: dropout-based simple graph contrastive learning for recommendation. Complex & Intelligent Systems (2023), 1–13.
  74. Graph contrastive learning with augmentations. NeurIPS 33 (2020), 5812–5823.
  75. Adaptive implicit friends identification over heterogeneous network for social recommendation. In CIKM. 357–366.
  76. XSimGCL: Towards Extremely Simple Graph Contrastive Learning for Recommendation. arXiv preprint arXiv:2209.02544 (2022).
  77. Are graph augmentations necessary? simple graph contrastive learning for recommendation. In SIGIR. 1294–1303.
  78. Solving the apparent diversity-accuracy dilemma of recommender systems. Proceedings of the National Academy of Sciences 107, 10 (2010), 4511–4515.
  79. Deep graph contrastive representation learning. arXiv preprint arXiv:2006.04131 (2020).
  80. Graph contrastive learning with adaptive augmentation. In TheWebConf (former WWW). 2069–2080.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (6)
  1. Jeongwhan Choi (24 papers)
  2. Hyowon Wi (10 papers)
  3. Chaejeong Lee (5 papers)
  4. Sung-Bae Cho (8 papers)
  5. Dongha Lee (63 papers)
  6. Noseong Park (78 papers)
Citations (1)
View on Semantic Scholar