Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

CPT: Competence-progressive Training Strategy for Few-shot Node Classification (2402.00450v3)

Published 1 Feb 2024 in cs.LG

Abstract: Graph Neural Networks (GNNs) have made significant advancements in node classification, but their success relies on sufficient labeled nodes per class in the training data. Real-world graph data often exhibits a long-tail distribution with sparse labels, emphasizing the importance of GNNs' ability in few-shot node classification, which entails categorizing nodes with limited data. Traditional episodic meta-learning approaches have shown promise in this domain, but they face an inherent limitation: it might lead the model to converge to suboptimal solutions because of random and uniform task assignment, ignoring task difficulty levels. This could lead the meta-learner to face complex tasks too soon, hindering proper learning. Ideally, the meta-learner should start with simple concepts and advance to more complex ones, like human learning. So, we introduce CPT, a novel two-stage curriculum learning method that aligns task difficulty with the meta-learner's progressive competence, enhancing overall performance. Specifically, in CPT's initial stage, the focus is on simpler tasks, fostering foundational skills for engaging with complex tasks later. Importantly, the second stage dynamically adjusts task difficulty based on the meta-learner's growing competence, aiming for optimal knowledge acquisition. Extensive experiments on popular node classification datasets demonstrate significant improvements of our strategy over existing methods.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (45)
  1. Curriculum learning. In Proceedings of the 26th Annual International Conference on Machine Learning, Jun 2009.
  2. Deep gaussian embedding of graphs: Unsupervised inductive learning via ranking. Cornell University - arXiv, Jul 2017.
  3. Curriculum labeling: Revisiting pseudo-labeling for semi-supervised learning. Proceedings of the AAAI Conference on Artificial Intelligence, page 6912–6920, Sep 2022.
  4. Few-shot learning on graphs via super-classes based on graph spectral measures. arXiv: Learning,arXiv: Learning, Feb 2020.
  5. Curriculum model adaptation with synthetic and real data for semantic foggy scene understanding. Cornell University - arXiv,Cornell University - arXiv, Jan 2019.
  6. Graph prototypical networks for few-shot learning on attributed networks. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management, Oct 2020.
  7. Meta-gnn on few-shot node classification in graph meta-learning. ACM Proceedings, Jan 2019.
  8. Semi-supervised semantic segmentation via dynamic self-training and class-balanced curriculum. ArXiv, abs/2004.08514, 2020.
  9. Model-agnostic meta-learning for fast adaptation of deep networks. arXiv: Learning, Mar 2017.
  10. Currmg: A curriculum learning approach for graph based molecular property prediction. In 2021 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), Dec 2021.
  11. Fine-tuning by curriculum learning for non-autoregressive neural machine translation. Proceedings of the AAAI Conference on Artificial Intelligence, page 7839–7846, Jun 2020.
  12. On the power of curriculum learning in training deep networks. Cornell University - arXiv, Apr 2019.
  13. Open graph benchmark: Datasets for machine learning on graphs. Cornell University - arXiv, May 2020.
  14. Tuneup: A training strategy for improving generalization of graph neural networks. arXiv preprint arXiv:2210.14843, 2022.
  15. Graph meta learning via local subgraphs. Cornell University - arXiv, Jan 2020.
  16. Self-paced learning with diversity. Neural Information Processing Systems, Dec 2014.
  17. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907, 2016.
  18. Curriculum graph machine learning: A survey. arXiv preprint arXiv:2302.02926, Feb 2023.
  19. Learning to propagate for graph meta-learning. Cornell University - arXiv,Cornell University - arXiv, Sep 2019.
  20. Adaptive task sampling for meta-learning. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XVIII 16, pages 752–769. Springer, 2020.
  21. Relative and absolute location embedding for few-shot node classification on graph. Proceedings of the … AAAI Conference on Artificial Intelligence, May 2021.
  22. Tail-gnn: Tail-node graph neural networks. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pages 1109–1119, 2021.
  23. Adaptive-step graph meta-learner for few-shot graph classification. Cornell University - arXiv,Cornell University - arXiv, Mar 2020.
  24. Inferring networks of substitutable and complementary products. arXiv: Social and Information Networks, Jun 2015.
  25. A simple neural attentive meta-learner. Learning,Learning, Jul 2017.
  26. Competence-based curriculum learning for neural machine translation. Cornell University - arXiv,Cornell University - arXiv, Mar 2019.
  27. Curriculum learning for heterogeneous star network embedding via deep reinforcement learning. In Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining, Feb 2018.
  28. Optimization as a model for few-shot learning. International Conference on Learning Representations,International Conference on Learning Representations, Apr 2017.
  29. Dropedge: Towards deep graph convolutional networks on node classification. Learning,Learning, Jul 2019.
  30. Prototypical networks for few-shot learning. Neural Information Processing Systems, Mar 2017.
  31. Meta-transfer learning for few-shot learning. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Jun 2019.
  32. Learning to compare: Relation network for few-shot learning. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Jun 2018.
  33. String v11: protein–protein association networks with increased coverage, supporting functional discovery in genome-wide experimental datasets. Nucleic acids research, 47(D1):D607–D613, 2019.
  34. Arnetminer. In Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining, Aug 2008.
  35. Generic and trend-aware curriculum learning for relation extraction. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2202–2213, 2022.
  36. Curriculum learning for graph neural networks: A multiview competence-based approach. arXiv preprint arXiv:2307.08859, 2023.
  37. Graph few-shot learning with attribute matching. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management, Oct 2020.
  38. Task-adaptive few-shot node classification. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 1910–1919, 2022.
  39. Curriculum pre-training heterogeneous subgraph transformer for top-n recommendation. ACM Transactions on Information Systems, page 1–28, Jan 2023.
  40. Clnode: Curriculum learning for node classification. In Proceedings of the Sixteenth ACM International Conference on Web Search and Data Mining, pages 670–678, 2023.
  41. A comprehensive survey on graph neural networks. IEEE Transactions on Neural Networks and Learning Systems, page 4–24, Jan 2021.
  42. How powerful are graph neural networks? arXiv preprint arXiv:1810.00826, 2018.
  43. Relation-aware heterogeneous graph for user profiling. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management, pages 3573–3577, 2021.
  44. Every document owns its structure: Inductive text classification via graph neural networks. arXiv preprint arXiv:2004.13826, 2020.
  45. Curriculum learning by dynamic instance hardness. Neural Information Processing Systems,Neural Information Processing Systems, Jan 2020.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (5)
  1. Qilong Yan (2 papers)
  2. Yufeng Zhang (67 papers)
  3. Jinghao Zhang (20 papers)
  4. Jingpu Duan (9 papers)
  5. Jian Yin (67 papers)

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now