2000 character limit reached
Overcoming Pitfalls in Graph Contrastive Learning Evaluation: Toward Comprehensive Benchmarks (2402.15680v1)
Published 24 Feb 2024 in cs.LG
Abstract: The rise of self-supervised learning, which operates without the need for labeled data, has garnered significant interest within the graph learning community. This enthusiasm has led to the development of numerous Graph Contrastive Learning (GCL) techniques, all aiming to create a versatile graph encoder that leverages the wealth of unlabeled data for various downstream tasks. However, the current evaluation standards for GCL approaches are flawed due to the need for extensive hyper-parameter tuning during pre-training and the reliance on a single downstream task for assessment. These flaws can skew the evaluation away from the intended goals, potentially leading to misleading conclusions. In our paper, we thoroughly examine these shortcomings and offer fresh perspectives on how GCL methods are affected by hyper-parameter choices and the choice of downstream tasks for their evaluation. Additionally, we introduce an enhanced evaluation framework designed to more accurately gauge the effectiveness, consistency, and overall capability of GCL methods.
- Graph barlow twins: A self-supervised representation learning framework for graphs. Knowledge-Based Systems 256 (2022), 109631.
- A simple framework for contrastive learning of visual representations. In International conference on machine learning. PMLR, 1597–1607.
- Graph transfer learning via adversarial domain adaptation with graph convolution. IEEE Transactions on Knowledge and Data Engineering 35, 5 (2022), 4908–4922.
- Convolutional networks on graphs for learning molecular fingerprints. Advances in neural information processing systems 28 (2015).
- Deep social collaborative filtering. In Proceedings of the 13th ACM RecSys.
- Simcse: Simple contrastive learning of sentence embeddings. arXiv preprint arXiv:2104.08821 (2021).
- Neural message passing for quantum chemistry. In International conference on machine learning. PMLR, 1263–1272.
- Dmitri Goldenberg. 2021. Social network analysis: From graph theory to applications with python. arXiv preprint arXiv:2102.10014 (2021).
- William L Hamilton. 2020. Graph representation learning. Morgan & Claypool Publishers.
- Kaveh Hassani and Amir Hosein Khasahmadi. 2020. Contrastive multi-view representation learning on graphs. In International conference on machine learning. PMLR, 4116–4126.
- Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 9729–9738.
- Molecular graph convolutions: moving beyond fingerprints. Journal of computer-aided molecular design 30 (2016), 595–608.
- Thomas N Kipf and Max Welling. 2016. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016).
- Gated graph sequence neural networks. arXiv preprint arXiv:1511.05493 (2015).
- Diffusion convolutional recurrent neural network: Data-driven traffic forecasting. arXiv preprint arXiv:1707.01926 (2017).
- Graph self-supervised learning: A survey. IEEE Transactions on Knowledge and Data Engineering 35, 6 (2022), 5879–5900.
- Learning disentangled representations for recommendation. Advances in neural information processing systems 32 (2019).
- Rethinking Sensors Modeling: Hierarchical Information Enhanced Traffic Forecasting. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management.
- Simple unsupervised graph representation learning. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 36. 7797–7805.
- Geometric matrix completion with recurrent multi-graph neural networks. Advances in neural information processing systems 30 (2017).
- Pitfalls of graph neural network evaluation. arXiv preprint arXiv:1811.05868 (2018).
- Large-scale representation learning on graphs via bootstrapping. arXiv preprint arXiv:2102.06514 (2021).
- Graph attention networks. arXiv preprint arXiv:1710.10903 (2017).
- Deep graph infomax. arXiv preprint arXiv:1809.10341 (2018).
- Graph wavenet for deep spatial-temporal graph modeling. arXiv preprint arXiv:1906.00121 (2019).
- How powerful are graph neural networks? arXiv preprint arXiv:1810.00826 (2018).
- Revisiting semi-supervised learning with graph embeddings. In International conference on machine learning. PMLR, 40–48.
- Graph convolutional neural networks for web-scale recommender systems. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining. 974–983.
- Graph convolutional policy network for goal-directed molecular graph generation. Advances in neural information processing systems 31 (2018).
- From canonical correlation analysis to self-supervised graph neural networks. Advances in Neural Information Processing Systems 34 (2021), 76–89.
- Adversarial label-flipping attack and defense for graph neural networks. In 2020 IEEE International Conference on Data Mining (ICDM). IEEE, 791–800.
- COSTA: covariance-preserving feature augmentation for graph contrastive learning. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 2524–2534.
- Spectral feature augmentation for graph contrastive learning and beyond. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 37. 11289–11297.
- Multi-label Node Classification On Graph-Structured Data. arXiv preprint arXiv:2304.10398 (2023).
- An empirical study of graph contrastive learning. arXiv preprint arXiv:2109.01116 (2021).
- Deep graph contrastive representation learning. arXiv preprint arXiv:2006.04131 (2020).
- Graph contrastive learning with adaptive augmentation. In Proceedings of the Web Conference 2021. 2069–2080.
- Marinka Zitnik and Jure Leskovec. 2017. Predicting multicellular function through multi-layer tissue networks. Bioinformatics 33, 14 (2017), i190–i198.