Papers
Topics
Authors
Recent
Detailed Answer
Quick Answer
Concise responses based on abstracts only
Detailed Answer
Well-researched responses based on abstracts and relevant paper content.
Custom Instructions Pro
Preferences or requirements that you'd like Emergent Mind to consider when generating responses
Gemini 2.5 Flash
Gemini 2.5 Flash 83 tok/s
Gemini 2.5 Pro 42 tok/s Pro
GPT-5 Medium 30 tok/s Pro
GPT-5 High 36 tok/s Pro
GPT-4o 108 tok/s Pro
Kimi K2 220 tok/s Pro
GPT OSS 120B 473 tok/s Pro
Claude Sonnet 4 39 tok/s Pro
2000 character limit reached

GCoder: Improving Large Language Model for Generalized Graph Problem Solving (2410.19084v1)

Published 24 Oct 2024 in cs.CL

Abstract: LLMs have demonstrated strong reasoning abilities, making them suitable for complex tasks such as graph computation. Traditional reasoning steps paradigm for graph problems is hindered by unverifiable steps, limited long-term reasoning, and poor generalization to graph variations. To overcome these limitations, we introduce GCoder, a code-based LLM designed to enhance problem-solving in generalized graph computation problems. Our method involves constructing an extensive training dataset, GraphWild, featuring diverse graph formats and algorithms. We employ a multi-stage training process, including Supervised Fine-Tuning (SFT) and Reinforcement Learning from Compiler Feedback (RLCF), to refine model capabilities. For unseen tasks, a hybrid retrieval technique is used to augment performance. Experiments demonstrate that GCoder outperforms GPT-4o, with an average accuracy improvement of 16.42% across various graph computational problems. Furthermore, GCoder efficiently manages large-scale graphs with millions of nodes and diverse input formats, overcoming the limitations of previous models focused on the reasoning steps paradigm. This advancement paves the way for more intuitive and effective graph problem-solving using LLMs. Code and data are available at here: https://github.com/Bklight999/WWW25-GCoder/tree/master.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (53)
  1. Extracting Small Subgraphs in Road Networks. In Proceedings of the ACM on Web Conference 2024. 493–502.
  2. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862 (2022).
  3. GraphQA: protein model quality assessment using graph convolutional networks. Bioinformatics 37, 3 (2021), 360–366.
  4. Graph of thoughts: Solving elaborate problems with large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 38. 17682–17690.
  5. Improving language models by retrieving from trillions of tokens. In International conference on machine learning. PMLR, 2206–2240.
  6. Fast and Accurate Fair k-Center Clustering in Doubling Metrics. In Proceedings of the ACM on Web Conference 2024. 756–767.
  7. Graphllm: Boosting graph reasoning ability of large language model. arXiv preprint arXiv:2310.05845 (2023).
  8. GraphWiz: An Instruction-Following Language Model for Graph Problems. arXiv preprint arXiv:2402.16029 (2024).
  9. Program of thoughts prompting: Disentangling computation from reasoning for numerical reasoning tasks. arXiv preprint arXiv:2211.12588 (2022).
  10. Link Recommendation to Augment Influence Diffusion with Provable Guarantees. In Proceedings of the ACM on Web Conference 2024. 2509–2518.
  11. Exploring the potential of large language models (llms) in learning on graphs. ACM SIGKDD Explorations Newsletter 25, 2 (2024), 42–61.
  12. Label-free Node Classification on Graphs with Large Language Models (LLMs). In The Twelfth International Conference on Learning Representations.
  13. How Do Large Language Models Understand Graph Patterns? A Benchmark for Graph Pattern Comprehension. arXiv preprint arXiv:2410.05298 (2024).
  14. MATHSENSEI: A Tool-Augmented Large Language Model for Mathematical Reasoning. arXiv preprint arXiv:2402.17231 (2024).
  15. A frustratingly simple approach improves textual graph learning. arXiv preprint arXiv:2308.02565 (2023).
  16. On the evolution of random graphs. Publ. math. inst. hung. acad. sci 5, 1 (1960), 17–60.
  17. Talk like a Graph: Encoding Graphs for Large Language Models. In NeurIPS 2023 Workshop: New Frontiers in Graph Learning.
  18. Critic: Large language models can self-correct with tool-interactive critiquing. arXiv preprint arXiv:2305.11738 (2023).
  19. Tora: A tool-integrated reasoning agent for mathematical problem solving. arXiv preprint arXiv:2309.17452 (2023).
  20. Gpt4graph: Can large language models understand graph structured data? an empirical evaluation and benchmarking. arXiv preprint arXiv:2305.15066 (2023).
  21. Exploring network structure, dynamics, and function using NetworkX. Technical Report. Los Alamos National Laboratory (LANL), Los Alamos, NM (United States).
  22. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685 (2021).
  23. Prompt-based node feature extractor for few-shot learning on text-attributed graphs. arXiv preprint arXiv:2309.02848 (2023).
  24. Llm-arc: Enhancing llms with an automated reasoning critic. arXiv preprint arXiv:2406.17663 (2024).
  25. LangChain-AI. [n. d.]. https://www.langchain.com/. Accessed: 2024-05-25.
  26. Can Large Language Models Analyze Graphs like Professionals? A Benchmark, Datasets and Models. arXiv preprint arXiv:2409.19667 (2024).
  27. VisionGraph: Leveraging Large Multimodal Models for Graph Theory Problems in Visual Context. arXiv preprint arXiv:2405.04950 (2024).
  28. A survey of graph meets large language model: Progress and future directions. arXiv preprint arXiv:2311.12399 (2023).
  29. Chang Liu and Bo Wu. 2023. Evaluating large language models on graphs: Performance insights and comparative analysis. arXiv preprint arXiv:2308.11224 (2023).
  30. GraphInstruct: Empowering Large Language Models with Graph Understanding and Reasoning Capability. arXiv preprint arXiv:2403.04483 (2024).
  31. Augmented language models: a survey. arXiv preprint arXiv:2302.07842 (2023).
  32. Local Centrality Minimization with Quality Guarantees. In Proceedings of the ACM on Web Conference 2024. 410–421.
  33. Finding Densest Subgraphs with Edge-Color Constraints. In Proceedings of the ACM on Web Conference 2024. 936–947.
  34. A Similarity-based Approach for Efficient Large Quasi-clique Detection. In Proceedings of the ACM on Web Conference 2024. 401–409.
  35. Talm: Tool augmented language models. arXiv preprint arXiv:2205.12255 (2022).
  36. Ryan A. Rossi and Nesreen K. Ahmed. 2015. The Network Data Repository with Interactive Graph Analytics and Visualization. In AAAI. https://networkrepository.com
  37. Enhancing retrieval-augmented large language models with iterative retrieval-generation synergy. arXiv preprint arXiv:2305.15294 (2023).
  38. Graphgpt: Graph instruction tuning for large language models. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval. 491–500.
  39. GraphArena: Benchmarking Large Language Models on Graph Computational Problems. arXiv preprint arXiv:2407.00379 (2024).
  40. A Fused Gromov-Wasserstein Framework for Unsupervised Knowledge Graph Entity Alignment. In Findings of the Association for Computational Linguistics: ACL 2023, Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (Eds.). Association for Computational Linguistics, Toronto, Canada, 3320–3334.
  41. Can Language Models Solve Graph Problems in Natural Language?. In Thirty-seventh Conference on Neural Information Processing Systems. https://openreview.net/forum?id=UDqHhbqYJV
  42. InstructGraph: Boosting Large Language Models via Graph-centric Instruction Tuning and Preference Alignment. arXiv preprint arXiv:2402.08785 (2024).
  43. Grapheval2000: Benchmarking and improving large language models on graph datasets. arXiv preprint arXiv:2406.16176 (2024).
  44. Opengraph: Towards open graph foundation models. arXiv preprint arXiv:2403.01121 (2024).
  45. C-pack: Packaged resources to advance general chinese embedding. arXiv preprint arXiv:2309.07597 (2023).
  46. Decomposition enhances reasoning via self-evaluation guided decoding. arXiv preprint arXiv:2305.00633 2 (2023).
  47. Evaluating spatial understanding of large language models. arXiv preprint arXiv:2310.14540 (2023).
  48. ReAct: Synergizing Reasoning and Acting in Language Models. In International Conference on Learning Representations (ICLR).
  49. GraCoRe: Benchmarking Graph Comprehension and Complex Reasoning in Large Language Models. arXiv preprint arXiv:2407.02936 (2024).
  50. Mammoth: Building math generalist models through hybrid instruction tuning. arXiv preprint arXiv:2309.05653 (2023).
  51. Can LLM Graph Reasoning Generalize beyond Pattern Memorization? arXiv preprint arXiv:2406.15992 (2024).
  52. LLM4DyG: Can Large Language Models Solve Problems on Dynamic Graphs? arXiv preprint arXiv:2310.17110 (2023).
  53. LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations). Association for Computational Linguistics, Bangkok, Thailand. http://arxiv.org/abs/2403.13372
List To Do Tasks Checklist Streamline Icon: https://streamlinehq.com

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Dice Question Streamline Icon: https://streamlinehq.com

Follow-Up Questions

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now