2000 character limit reached
Graph Neural Architecture Search with GPT-4 (2310.01436v2)
Published 30 Sep 2023 in cs.LG and cs.AI
Abstract: Graph Neural Architecture Search (GNAS) has shown promising results in automatically designing graph neural networks. However, GNAS still requires intensive human labor with rich domain knowledge to design the search space and search strategy. In this paper, we integrate GPT-4 into GNAS and propose a new GPT-4 based Graph Neural Architecture Search method (GPT4GNAS for short). The basic idea of our method is to design a new class of prompts for GPT-4 to guide GPT-4 toward the generative task of graph neural architectures. The prompts consist of descriptions of the search space, search strategy, and search feedback of GNAS. By iteratively running GPT-4 with the prompts, GPT4GNAS generates more accurate graph neural networks with fast convergence. Experimental results show that embedding GPT-4 into GNAS outperforms the state-of-the-art GNAS methods.
- 2023. Chatgpt participates in a computer science exam. ArXiv preprint abs/2303.09461.
- 2021. Diffmg: Differentiable meta graph search for heterogeneous graph neural networks. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, 279–288.
- 2020. Graph neural architecture search. In Bessiere, C., ed., Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI 2020, 1403–1409. ijcai.org.
- 2021. Heterogeneous graph neural architecture search. In 2021 IEEE International Conference on Data Mining (ICDM), 1066–1071. IEEE.
- 2022. Graphnas++: Distributed architecture search for graph neural networks. IEEE Transactions on Knowledge and Data Engineering.
- 2023. Hgnas++: efficient architecture search for heterogeneous graph neural networks. IEEE Transactions on Knowledge and Data Engineering.
- 2022. Large-scale graph neural architecture search. In Chaudhuri, K.; Jegelka, S.; Song, L.; Szepesvári, C.; Niu, G.; and Sabato, S., eds., International Conference on Machine Learning, ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA, volume 162 of Proceedings of Machine Learning Research, 7968–7981. PMLR.
- 2023. Gpt4graph: Can large language models understand graph structured data? an empirical evaluation and benchmarking. ArXiv preprint abs/2305.15066.
- 2021. Search to aggregate neighborhood for graph neural network. In 2021 IEEE 37th International Conference on Data Engineering (ICDE), 552–563. IEEE.
- 2021. Graph-enhanced multi-task learning of multi-level transition dynamics for session-based recommendation. In Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Artificial Intelligence, EAAI 2021, Virtual Event, February 2-9, 2021, 4123–4130. AAAI Press.
- 2020. Autograph: Automated graph neural network. In Neural Information Processing: 27th International Conference, ICONIP 2020, Bangkok, Thailand, November 23–27, 2020, Proceedings, Part II 27, 189–201. Springer.
- 2021. One-shot graph neural architecture search with dynamic search space. In Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Artificial Intelligence, EAAI 2021, Virtual Event, February 2-9, 2021, 8510–8517. AAAI Press.
- 2023. Gpteval: Nlg evaluation using gpt-4 with better human alignment. ArXiv preprint abs/2303.16634.
- 2021. Graph neural architecture search: A survey. Tsinghua Science and Technology.
- OpenAI. 2023. Gpt-4 technical report.
- 2022a. Graph neural architecture search under distribution shifts. In Chaudhuri, K.; Jegelka, S.; Song, L.; Szepesvári, C.; Niu, G.; and Sabato, S., eds., International Conference on Machine Learning, ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA, volume 162 of Proceedings of Machine Learning Research, 18083–18095. PMLR.
- 2022b. Nas-bench-graph: Benchmarking graph neural architecture search. Advances in Neural Information Processing Systems 35:54–69.
- 2022. Genetic-gnn: Evolutionary architecture search for graph neural networks. Knowledge-Based Systems.
- 2021. Autogel: An automated graph neural network with explicit link information. In Ranzato, M.; Beygelzimer, A.; Dauphin, Y. N.; Liang, P.; and Vaughan, J. W., eds., Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual, 24509–24522.
- 2022. Autogt: Automated graph transformer architecture search. In The Eleventh International Conference on Learning Representations.
- 2023a. Controllable text-to-image generation with gpt-4. ArXiv preprint abs/2305.18583.
- 2023b. Dynamic heterogeneous graph attention neural architecture search. In Thirty-Seventh AAAI Conference on Artificial Intelligence, AAAI 2023, Thirty-Fifth Conference on Innovative Applications of Artificial Intelligence, IAAI 2023, Thirteenth Symposium on Educational Advances in Artificial Intelligence, EAAI 2023, Washington, DC, USA, February 7-14, 2023, 11307–11315. AAAI Press.
- 2021. Automated machine learning on graphs: A survey. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI 2021, Virtual Event / Montreal, Canada, 19-27 August 2021, 4704–4712. ijcai.org.
- Zhang, J. 2023. Graph-toolformer: To empower llms with graph reasoning ability via prompt augmented by chatgpt. arXiv preprint arXiv:2304.11116.
- 2022. Multi-relational graph neural architecture search with fine-grained message passing. 2022 IEEE International Conference on Data Mining (ICDM).
- 2023a. Can gpt-4 perform neural architecture search? ArXiv preprint abs/2304.10970.
- 2023b. Auto-heg: Automated graph neural network on heterophilic graphs. In Proceedings of the ACM Web Conference 2023.
- 2022. Auto-gnn: Neural architecture search of graph neural networks. Frontiers in big Data.
Collections
Sign up for free to add this paper to one or more collections.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.