Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
41 tokens/sec
GPT-4o
60 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
8 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Exploring the Potential of Large Language Models in Graph Generation (2403.14358v1)

Published 21 Mar 2024 in cs.LG, cs.AI, and q-bio.BM

Abstract: LLMs have achieved great success in many fields, and recent works have studied exploring LLMs for graph discriminative tasks such as node classification. However, the abilities of LLMs for graph generation remain unexplored in the literature. Graph generation requires the LLM to generate graphs with given properties, which has valuable real-world applications such as drug discovery, while tends to be more challenging. In this paper, we propose LLM4GraphGen to explore the ability of LLMs for graph generation with systematical task designs and extensive experiments. Specifically, we propose several tasks tailored with comprehensive experiments to address key questions regarding LLMs' understanding of different graph structure rules, their ability to capture structural type distributions, and their utilization of domain knowledge for property-based graph generation. Our evaluations demonstrate that LLMs, particularly GPT-4, exhibit preliminary abilities in graph generation tasks, including rule-based and distribution-based generation. We also observe that popular prompting methods, such as few-shot and chain-of-thought prompting, do not consistently enhance performance. Besides, LLMs show potential in generating molecules with specific properties. These findings may serve as foundations for designing good LLMs based models for graph generation and provide valuable insights and further research.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (41)
  1. A survey of large language models. arXiv preprint arXiv:2303.18223, 2023.
  2. On the opportunities and risks of foundation models. arXiv preprint arXiv:2303.18223, 2022.
  3. Language models are few-shot learners. In Advances in Neural Information Processing Systems, pages 1877–1901, 2020.
  4. Sparks of artificial general intelligence: Early experiments with gpt-4. arXiv preprint arXiv:2303.12712, 2023.
  5. Chain-of-thought prompting elicits reasoning in large language models. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems, volume 35, pages 24824–24837, 2022.
  6. LEVER: Learning to verify language-to-code generation with execution. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett, editors, Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages 26106–26128. PMLR, 2023. URL https://proceedings.mlr.press/v202/ni23b.html.
  7. Expectation vs. experience: Evaluating the usability of code generation tools powered by large language models. In Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems, CHI EA ’22, New York, NY, USA, 2022. Association for Computing Machinery. ISBN 9781450391566. doi:10.1145/3491101.3519665. URL https://doi.org/10.1145/3491101.3519665.
  8. LLM drug discovery challenge: A contest as a feasibility study on the utilization of large language models in medicinal chemistry. In AI for Accelerated Materials Design - NeurIPS 2023 Workshop, 2023. URL https://openreview.net/forum?id=kjUylvko18.
  9. Artificial intelligence enabled chatgpt and large language models in drug target discovery, drug discovery, and development. Molecular Therapy-Nucleic Acids, 33:866–868, 2023.
  10. Llm-assisted knowledge graph engineering: Experiments with chatgpt. arXiv preprint arXiv:2307.06917, 2023.
  11. Head-to-tail: How knowledgeable are large language models (llm)? aka will llms replace knowledge graphs? arXiv preprint arXiv:2308.10168, 2023.
  12. Graph meets llms: Towards large graph models. NeurIPS 2023 New Frontiers in Graph Learning Workshop, 2023a.
  13. Can language models solve graph problems in natural language? Thirty-seventh Conference on Neural Information Processing Systems, 2023a.
  14. Natural language is all a graph needs. arXiv preprint arXiv:2308.07134, 2023.
  15. Can llms effectively leverage graph structural information: When and why. arXiv preprint arXiv:2309.16595, 2023.
  16. Graphgpt: Graph instruction tuning for large language models. arXiv preprint arXiv:2310.13023, 2023.
  17. A survey on deep graph generation: Methods and applications. In Learning on Graphs Conference, pages 47–1, 2022.
  18. Graph convolutional policy network for goal-directed molecular graph generation. Advances in neural information processing systems, 31, 2018.
  19. Gnnexplainer: Generating explanations for graph neural networks. Advances in neural information processing systems, 32, 2019.
  20. Discovering invariant rationales for graph neural networks. In International Conference on Learning Representations, 2022.
  21. Graph neural architecture search under distribution shifts. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvári, Gang Niu, and Sivan Sabato, editors, International Conference on Machine Learning, ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA, volume 162 of Proceedings of Machine Learning Research, pages 18083–18095. PMLR, 2022. URL https://proceedings.mlr.press/v162/qin22b.html.
  22. A systematic survey on deep generative models for graph generation. IEEE Trans. Pattern Anal. Mach. Intell., 45(5):5370–5390, 2023. doi:10.1109/TPAMI.2022.3214832. URL https://doi.org/10.1109/TPAMI.2022.3214832.
  23. Open graph benchmark: Datasets for machine learning on graphs. Advances in neural information processing systems, 33:22118–22133, 2020.
  24. David Weininger. Smiles, a chemical language and information system. 1. introduction to methodology and encoding rules. Journal of chemical information and computer sciences, 28(1):31–36, 1988.
  25. Explanations as features: Llm-based features for text-attributed graphs. arXiv preprint arXiv:2305.19523, 2023.
  26. Exploring the potential of large language models (llms) in learning on graphs. arXiv preprint arXiv:2307.03393, 2023.
  27. Gpt4graph: Can large language models understand graph structured data? an empirical evaluation and benchmarking. arXiv preprint arXiv:2305.15066, 2023.
  28. Llm4dyg: Can large language models solve problems on dynamic graphs? arXiv preprint arXiv:2310.17110, 2023b.
  29. Jiawei Zhang. Graph-toolformer: To empower llms with graph reasoning ability via prompt augmented by chatgpt. arXiv preprint arXiv:2304.11116, 2023.
  30. Structgpt: A general framework for large language model to reason over structured data. arXiv preprint arXiv:2305.09645, 2023.
  31. Unifying large language models and knowledge graphs: A roadmap. arXiv preprint arXiv:2306.08302, 2023.
  32. Can language models solve graph problems in natural language? CoRR, abs/2305.10037, 2023b. doi:10.48550/arXiv.2305.10037. URL https://doi.org/10.48550/arXiv.2305.10037.
  33. Graphgen-redux: a fast and lightweight recurrent model for labeled graph generation. In International Joint Conference on Neural Networks, IJCNN 2021, Shenzhen, China, July 18-22, 2021, pages 1–8. IEEE, 2021. doi:10.1109/IJCNN52387.2021.9533743. URL https://doi.org/10.1109/IJCNN52387.2021.9533743.
  34. Interpretable Molecular Graph Generation via Monotonic Constraints, pages 73–81. doi:10.1137/1.9781611977172.9. URL https://epubs.siam.org/doi/abs/10.1137/1.9781611977172.9.
  35. Junction tree variational autoencoder for molecular graph generation. In Jennifer G. Dy and Andreas Krause, editors, Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässan, Stockholm, Sweden, July 10-15, 2018, volume 80 of Proceedings of Machine Learning Research, pages 2328–2337. PMLR, 2018. URL http://proceedings.mlr.press/v80/jin18a.html.
  36. Moflow: An invertible flow model for generating molecular graphs. In Rajesh Gupta, Yan Liu, Jiliang Tang, and B. Aditya Prakash, editors, KDD ’20: The 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Virtual Event, CA, USA, August 23-27, 2020, pages 617–626. ACM, 2020. doi:10.1145/3394486.3403104. URL https://doi.org/10.1145/3394486.3403104.
  37. Graphdf: A discrete flow model for molecular graph generation. In Marina Meila and Tong Zhang, editors, Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event, volume 139 of Proceedings of Machine Learning Research, pages 7192–7203. PMLR, 2021. URL http://proceedings.mlr.press/v139/luo21a.html.
  38. Multi-motifgan (MMGAN): motif-targeted graph generation and prediction. In 2020 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2020, Barcelona, Spain, May 4-8, 2020, pages 4182–4186. IEEE, 2020. doi:10.1109/ICASSP40776.2020.9053451. URL https://doi.org/10.1109/ICASSP40776.2020.9053451.
  39. Mol-cyclegan: a generative model for molecular optimization. J. Cheminformatics, 12(1):2, 2020. doi:10.1186/S13321-019-0404-1. URL https://doi.org/10.1186/s13321-019-0404-1.
  40. Score-based generative modeling of graphs via the system of stochastic differential equations. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvári, Gang Niu, and Sivan Sabato, editors, International Conference on Machine Learning, ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA, volume 162 of Proceedings of Machine Learning Research, pages 10362–10383. PMLR, 2022. URL https://proceedings.mlr.press/v162/jo22a.html.
  41. Digress: Discrete denoising diffusion for graph generation. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023. URL https://openreview.net/pdf?id=UaAD-Nu86WX.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (9)
  1. Yang Yao (17 papers)
  2. Xin Wang (1306 papers)
  3. Zeyang Zhang (28 papers)
  4. Yijian Qin (9 papers)
  5. Ziwei Zhang (40 papers)
  6. Xu Chu (66 papers)
  7. Yuekui Yang (10 papers)
  8. Wenwu Zhu (104 papers)
  9. Hong Mei (15 papers)
Citations (10)
View on Semantic Scholar