Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
110 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Adversarial Curriculum Graph Contrastive Learning with Pair-wise Augmentation (2402.10468v1)

Published 16 Feb 2024 in cs.LG and cs.AI

Abstract: Graph contrastive learning (GCL) has emerged as a pivotal technique in the domain of graph representation learning. A crucial aspect of effective GCL is the caliber of generated positive and negative samples, which is intrinsically dictated by their resemblance to the original data. Nevertheless, precise control over similarity during sample generation presents a formidable challenge, often impeding the effective discovery of representative graph patterns. To address this challenge, we propose an innovative framework: Adversarial Curriculum Graph Contrastive Learning (ACGCL), which capitalizes on the merits of pair-wise augmentation to engender graph-level positive and negative samples with controllable similarity, alongside subgraph contrastive learning to discern effective graph patterns therein. Within the ACGCL framework, we have devised a novel adversarial curriculum training methodology that facilitates progressive learning by sequentially increasing the difficulty of distinguishing the generated samples. Notably, this approach transcends the prevalent sparsity issue inherent in conventional curriculum learning strategies by adaptively concentrating on more challenging training data. Finally, a comprehensive assessment of ACGCL is conducted through extensive experiments on six well-known benchmark datasets, wherein ACGCL conspicuously surpasses a set of state-of-the-art baselines.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (57)
  1. Curriculum learning. In Proceedings of the 26th annual international conference on machine learning. 41–48.
  2. A simple framework for contrastive learning of visual representations. In International conference on machine learning. PMLR, 1597–1607.
  3. CuCo: Graph Representation with Curriculum Contrastive Learning.. In IJCAI. 2300–2306.
  4. Debiased contrastive learning. Advances in neural information processing systems 33 (2020), 8765–8775.
  5. Marco Cuturi and Arnaud Doucet. 2014. Fast computation of Wasserstein barycenters. In International conference on machine learning. PMLR, 685–693.
  6. Student-teacher curriculum learning via reinforcement learning: predicting hospital inpatient admission location. In International Conference on Machine Learning. PMLR, 2848–2857.
  7. Reverse curriculum generation for reinforcement learning. In Conference on robot learning. PMLR, 482–495.
  8. A survey of graph neural networks for recommender systems: Challenges, methods, and directions. ACM Transactions on Recommender Systems 1, 1 (2023), 1–51.
  9. Decomposition-based evolutionary multiobjective optimization to self-paced learning. IEEE Transactions on Evolutionary Computation 23, 2 (2018), 288–302.
  10. Bi-directional Heterogeneous Graph Hashing towards Efficient Outfit Recommendation. In Proceedings of the 30th ACM International Conference on Multimedia. 268–276.
  11. Guy Hacohen and Daphna Weinshall. 2019. On the power of curriculum learning in training deep networks. In International Conference on Machine Learning. PMLR, 2535–2544.
  12. Inductive representation learning on large graphs. Advances in neural information processing systems 30 (2017).
  13. Kaveh Hassani and Amir Hosein Khasahmadi. 2020. Contrastive multi-view representation learning on graphs. In International Conference on Machine Learning. PMLR, 4116–4126.
  14. Learning deep representations by mutual information estimation and maximization. arXiv preprint arXiv:1808.06670 (2018).
  15. Easy samples first: Self-paced reranking for zero-example multimedia search. In Proceedings of the 22nd ACM international conference on Multimedia. 547–556.
  16. Sub-graph contrast for scalable self-supervised graph representation learning. In 2020 IEEE International Conference on Data Mining (ICDM). IEEE, 222–231.
  17. Supervised contrastive learning. Advances in Neural Information Processing Systems 33 (2020), 18661–18673.
  18. Thomas N Kipf and Max Welling. 2016. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016).
  19. Predict then propagate: Graph neural networks meet personalized pagerank. arXiv preprint arXiv:1810.05997 (2018).
  20. Self-paced learning for latent variable models. Advances in neural information processing systems 23 (2010).
  21. Self-attention graph pooling. In International conference on machine learning. PMLR, 3734–3743.
  22. Augmentation-free self-supervised learning on graphs. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 36. 7372–7380.
  23. Lu Lin and Jinghui Chen. 2023. Spectral Augmentation for Self-Supervised Learning on Graphs. In The Eleventh International Conference on Learning Representations.
  24. Prototypical graph contrastive learning. IEEE Transactions on Neural Networks and Learning Systems (2022).
  25. Revisiting graph contrastive learning from the perspective of graph spectrum. arXiv preprint arXiv:2210.02330 (2022).
  26. Learning fair node representations with graph counterfactual fairness. In Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining. 695–703.
  27. Simple unsupervised graph representation learning. AAAI.
  28. Graph representation learning via graphical mutual information maximization. In Proceedings of The Web Conference 2020. 259–270.
  29. Gustavo Penha and Claudia Hauff. 2019. Curriculum learning strategies for IR: An empirical study on conversation response ranking. arXiv preprint arXiv:1912.08555 (2019).
  30. Gcc: Graph contrastive coding for graph neural network pre-training. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 1150–1160.
  31. Facenet: A unified embedding for face recognition and clustering. In Proceedings of the IEEE conference on computer vision and pattern recognition. 815–823.
  32. Pitfalls of graph neural network evaluation. arXiv preprint arXiv:1811.05868 (2018).
  33. A graph to graphs framework for retrosynthesis prediction. In International conference on machine learning. PMLR, 8818–8827.
  34. Martin Simonovsky and Nikos Komodakis. 2018. Graphvae: Towards generation of small graphs using variational autoencoders. In International conference on artificial neural networks. Springer, 412–422.
  35. Curriculum learning: A survey. International Journal of Computer Vision 130, 6 (2022), 1526–1565.
  36. Arnetminer: extraction and mining of academic social networks. In Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining. 990–998.
  37. Large-scale representation learning on graphs via bootstrapping. arXiv preprint arXiv:2102.06514 (2021).
  38. Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of machine learning research 9, 11 (2008).
  39. Deep Graph Infomax. ICLR (Poster) 2, 3 (2019), 4.
  40. A survey on curriculum learning. IEEE Transactions on Pattern Analysis and Machine Intelligence (2021).
  41. MMGCN: Multi-modal graph convolution network for personalized recommendation of micro-video. In Proceedings of the 27th ACM International Conference on Multimedia. 1437–1445.
  42. Graph neural networks in recommender systems: a survey. Comput. Surveys 55, 5 (2022), 1–37.
  43. Session-based recommendation with graph neural networks. In Proceedings of the AAAI conference on artificial intelligence, Vol. 33. 346–353.
  44. ProGCL: Rethinking Hard Negative Mining in Graph Contrastive Learning. In International Conference on Machine Learning. PMLR, 24332–24346.
  45. Understanding negative sampling in graph representation learning. In Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining. 1666–1676.
  46. Autogcl: Automated graph contrastive learning via learnable view generators. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 36. 8892–8900.
  47. Graph convolutional neural networks for web-scale recommender systems. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining. 974–983.
  48. Graph contrastive learning with augmentations. Advances in Neural Information Processing Systems 33 (2020), 5812–5823.
  49. Bringing Your Own View: Graph Contrastive Learning without Prefabricated Data Augmentations. arXiv preprint arXiv:2201.01702 (2022).
  50. Muhan Zhang and Yixin Chen. 2018. Link prediction based on graph neural networks. Advances in neural information processing systems 31 (2018).
  51. Motif-driven contrastive learning of graph representations. arXiv preprint arXiv:2012.12533 (2020).
  52. Self-paced learning for matrix factorization. In Twenty-ninth AAAI conference on artificial intelligence.
  53. Counterfactual Graph Learning for Link Prediction. arXiv preprint arXiv:2106.02172 (2021).
  54. Embedding in Recommender Systems: A Survey. arXiv preprint arXiv:2310.18608 (2023).
  55. RoSA: A Robust Self-Aligned Framework for Node-Node Graph Contrastive Learning. arXiv preprint arXiv:2204.13846 (2022).
  56. Deep graph contrastive representation learning. arXiv preprint arXiv:2006.04131 (2020).
  57. Graph contrastive learning with adaptive augmentation. In Proceedings of the Web Conference 2021. 2069–2080.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (5)
  1. Xinjian Zhao (8 papers)
  2. Liang Zhang (357 papers)
  3. Yang Liu (2253 papers)
  4. Ruocheng Guo (62 papers)
  5. Xiangyu Zhao (192 papers)

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets