All in One: Multi-Task Prompting for Graph Neural Networks (Extended Abstract) (2403.07040v1)
Abstract: This paper is an extended abstract of our original work published in KDD23, where we won the best research paper award (Xiangguo Sun, Hong Cheng, Jia Li, Bo Liu, and Jihong Guan. All in one: Multi-task prompting for graph neural networks. KDD 23) The paper introduces a novel approach to bridging the gap between pre-trained graph models and the diverse tasks they're applied to, inspired by the success of prompt learning in NLP. Recognizing the challenge of aligning pre-trained models with varied graph tasks (node level, edge level, and graph level), which can lead to negative transfer and poor performance, we propose a multi-task prompting method for graphs. This method involves unifying graph and language prompt formats, enabling NLP's prompting strategies to be adapted for graph tasks. By analyzing the task space of graph applications, we reformulate problems to fit graph-level tasks and apply meta-learning to improve prompt initialization for multiple tasks. Experiments show our method's effectiveness in enhancing model performance across different graph tasks. Beyond the original work, in this extended abstract, we further discuss the graph prompt from a bigger picture and provide some of the latest work toward this area.
- Multi-level graph convolutional networks for cross-platform anchor link prediction. In Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, pages 1503–1511, 2020.
- Prompt learning on temporal interaction graphs. arXiv:2402.06326, 2024.
- Dynamic recommendation based on graph diffusion and ebbinghaus curve. IEEE Transactions on Computational Social Systems, 2023.
- Prompt tuning for graph neural networks. arXiv preprint arXiv:2209.15240, 2022.
- Protein multimer structure prediction via PPI-guided prompt learning. In The Twelfth International Conference on Learning Representations (ICLR), 2024.
- Inductive representation learning on large graphs. Advances in neural information processing systems, 30, 2017.
- Self-supervised learning on graphs: Deep insights and new direction. arXiv preprint arXiv:2006.10141, 2020.
- Attention is not the only choice: Counterfactual reasoning for path-based explainable recommendation. IEEE Transactions on Knowledge and Data Engineering, 2024.
- A survey of graph meets large language model: Progress and future directions. arXiv:2311.12399, 2024.
- Nowhere to hide: Online rumor detection based on retweeting graph neural networks. IEEE Transactions on Neural Networks and Learning Systems, 2022.
- Recognize news transition from collective behavior for news recommendation. ACM Transactions on Information Systems, 41(4):1–30, 2023.
- Computing graph edit distance via neural graph matching. Proceedings of the VLDB Endowment, 16(8):1817–1829, 2023.
- Pitfalls of graph neural network evaluation. arXiv preprint arXiv:1811.05868, 2018.
- Towards out-of-distribution generalization: A survey. arXiv preprint arXiv:2108.13624, 2021.
- Heterogeneous hypergraph embedding for graph classification. In Proceedings of the 14th acm international conference on web search and data mining, pages 725–733, 2021.
- Multi-level hyperedge distillation for social linking prediction on sparsely observed networks. In Proceedings of the Web Conference 2021, pages 2934–2945, 2021.
- GPPT: Graph pre-training and prompt tuning to generalize graph neural networks. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 1717–1727, 2022.
- In your eyes: Modality disentangling for personality analysis in short video. IEEE Transactions on Computational Social Systems, 2022.
- Structure learning via meta-hyperedge for dynamic rumor detection. IEEE Transactions on Knowledge and Data Engineering, 2022.
- Counter-empirical attacking based on adversarial reinforcement learning for time-relevant scoring system. IEEE Transactions on Knowledge and Data Engineering, 2023.
- All in one: Multi-task prompting for graph neural networks. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 2120–2131, 2023.
- Self-supervised hypergraph representation learning for sociological analysis. IEEE Transactions on Knowledge and Data Engineering, 2023.
- Graph prompt learning: A comprehensive survey and beyond. arXiv:2311.16534, 2023.
- Semi-supervised classification with graph convolutional networks. In J. International Conference on Learning Representations (ICLR 2017), 2016.
- Simgrace: A simple framework for graph contrastive learning without data augmentation. In Proceedings of the ACM Web Conference 2022, pages 1070–1079, 2022.
- Generating counterfactual hard negative samples for graph contrastive learning. In Proceedings of the ACM Web Conference 2023, pages 621–629, 2023.
- Graph contrastive learning with augmentations. Advances in Neural Information Processing Systems, 33:5812–5823, 2020.
- Unsupervised graph poisoning attack via contrastive loss back-propagation. In Proceedings of the ACM Web Conference 2022, pages 1322–1330, 2022.
- Graph masked autoencoders with transformers. arXiv preprint arXiv:2202.08391, 2022.
- All in one and one for all: A simple yet effective method towards cross-domain graph pretraining. 2024.