Curriculum Graph Machine Learning: A Survey (2302.02926v2)
Abstract: Graph machine learning has been extensively studied in both academia and industry. However, in the literature, most existing graph machine learning models are designed to conduct training with data samples in a random order, which may suffer from suboptimal performance due to ignoring the importance of different graph data samples and their training orders for the model optimization status. To tackle this critical problem, curriculum graph machine learning (Graph CL), which integrates the strength of graph machine learning and curriculum learning, arises and attracts an increasing amount of attention from the research community. Therefore, in this paper, we comprehensively overview approaches on Graph CL and present a detailed survey of recent advances in this direction. Specifically, we first discuss the key challenges of Graph CL and provide its formal problem definition. Then, we categorize and summarize existing methods into three classes based on three kinds of graph machine learning tasks, i.e., node-level, link-level, and graph-level tasks. Finally, we share our thoughts on future research directions. To the best of our knowledge, this paper is the first survey for curriculum graph machine learning.
- Hyper-graph-based attention curriculum learning using a lexical algorithm for mental health. Pattern Recognition Letters, 2022.
- The traveling salesman problem. In The Traveling Salesman Problem. Princeton university press, 2011.
- Curriculum learning. In ICML, 2009.
- Time series analysis: forecasting and control. John Wiley & Sons, 2015.
- Superloss: A generic loss for robust curriculum learning. NeurIPS, 2020.
- Cuco: Graph representation with curriculum contrastive learning. In IJCAI, 2021.
- A survey on network embedding. IEEE TKDE, 2018.
- Graph analysis of functional brain networks: practical issues in translational neuroscience. Philosophical Transactions of the Royal Society B: Biological Sciences, 2014.
- Dual graph convolutional networks with transformer and curriculum learning for image captioning. In ACM Multimedia, 2021.
- Jeffrey L Elman. Learning and development in neural networks: The importance of starting small. Cognition, 1993.
- Event classification in microblogs via social tracking. ACM TIST, 2017.
- Predict then propagate: Graph neural networks meet personalized pagerank. arXiv:1810.05997, 2018.
- Neural message passing for quantum chemistry. In ICML, 2017.
- Why curriculum learning & self-paced learning work in big/noisy data: A theoretical perspective. Big Data and Information Analytics, 2016.
- Multi-modal curriculum learning over graphs. ACM TIST, 2019.
- An efficient curriculum learning-based strategy for molecular graph learning. Briefings in Bioinformatics, 2022.
- Large-scale graph neural architecture search. In ICML, 2022.
- Curriculumnet: Weakly supervised learning from large-scale web images. In ECCV, 2018.
- On the power of curriculum learning in training deep networks. In ICML, 2019.
- Inductive representation learning on large graphs. NeurIPS, 2017.
- Dorit S Hochba. Approximation algorithms for np-hard problems. ACM Sigact News, 1997.
- Covid-19 detection through transfer learning using multimodal imaging data. Ieee Access, 2020.
- Drug repurposing for covid-19 using graph neural network and harmonizing multiple evidence. Scientific reports, 2021.
- Open graph benchmark: Datasets for machine learning on graphs. NeurIPS, 2020.
- Strategies for pre-training graph neural networks. In ICLR, 2020.
- Tuneup: A training strategy for improving generalization of graph neural networks. arXiv:2210.14843, 2022.
- Label propagation for deep semi-supervised learning. In CVPR, 2019.
- Easy samples first: Self-paced reranking for zero-example multimedia search. In ACM Multimedia, 2014.
- Semi-supervised classification with graph convolutional networks. In ICLR, 2017.
- Ood-gnn: Out-of-distribution generalized graph neural network. IEEE TKDE, 2022.
- Out-of-distribution generalization on graphs: A survey. arXiv:2202.07987, 2022.
- Graph neural network with curriculum learning for imbalanced node classification. arXiv:2202.02529, 2022.
- Hard sample aware network for contrastive deep graph clustering. AAAI, 2023.
- Efficient neural architecture search via parameters sharing. In ICML, 2018.
- Competence-based curriculum learning for neural machine translation. arXiv:1903.09848, 2019.
- Martin L Puterman. Markov decision processes. Handbooks in operations research and management science, 1990.
- Curriculum learning for heterogeneous star network embedding via deep reinforcement learning. In WSDM, 2018.
- Language acquisition in the absence of explicit negative evidence: How important is starting small? Cognition, 1999.
- Curriculum graph co-teaching for multi-target domain adaptation. In CVPR, 2021.
- Learning to simulate complex physics with graph networks. In ICML, 2020.
- Curriculum learning: A survey. IJCV, 2022.
- Ranking-based clustering of heterogeneous information networks with star network schema. In SIGKDD, 2009.
- Pathsim: Meta path-based top-k similarity search in heterogeneous information networks. VLDB Endowment, 2011.
- Infograph: Unsupervised and semi-supervised graph-level representation learning via mutual information maximization. In ICLR, 2019.
- Simple and effective curriculum pointer-generator networks for reading comprehension over long narratives. arXiv:1905.10847, 2019.
- Importance sampling: a review. Wiley Interdisciplinary Reviews: Computational Statistics, 2010.
- Digraph inception convolutional networks. NeurIPS, 2020.
- Directed graph contrastive learning. NeurIPS, 2021.
- Generic and trend-aware curriculum learning for relation extraction in graph neural networks. arXiv:2205.08625, 2022.
- Graph attention networks. In ICLR, 2018.
- Knowledge graph embedding: A survey of approaches and applications. IEEE TKDE, 2017.
- Community preserving network embedding. In AAAI, 2017.
- Dynamic curriculum learning for imbalanced data classification. In ICCV, 2019.
- A survey on curriculum learning. IEEE TPAMI, 2021.
- Curgraph: Curriculum learning for graph classification. In The WebConf, 2021.
- Curriculum pre-training heterogeneous subgraph transformer for top-n recommendation. TOIS, 2023.
- Clnode: Curriculum learning for node classification. arXiv:2206.07258, 2022.
- Curriculum learning by transfer learning: Theory and experiments with deep networks. In ICML, 2018.
- A comprehensive survey on graph neural networks. IEEE TNNLS, 2020.
- Connecting the dots: Multivariate time series forecasting with graph neural networks. In SIGKDD, 2020.
- Graph neural networks in recommender systems: a survey. ACM Computing Surveys, 2022.
- How powerful are graph neural networks? ICLR, 2019.
- Graph convolutional neural networks for web-scale recommender systems. In SIGKDD, 2018.
- Graph contrastive learning with augmentations. NeurIPS, 2020.
- Si Zhang and Hanghang Tong. Final: Fast attributed network alignment. In SIGKDD, 2016.
- Deep learning on graphs: A survey. IEEE TKDE, 2020.
- Few-shot learning on graphs: A survey. IJCAI, 2022.
- Learning to solve travelling salesman problem with hardness-adaptive curriculum. In AAAI, 2022.
- Curriculum learning for graph neural networks: Which edges should we learn first. NeurIPS, 2023.
- Mentorgnn: deriving curriculum for pre-training gnns. In CIKM, 2022.
- Curml: A curriculum machine learning library. In ACM Multimedia, 2022.
- Shift-robust gnns: Overcoming the limitations of localized graph training data. NeurIPS, 2021.