Papers
Topics
Authors
Recent
Search
2000 character limit reached

UrbanKGent: A Unified Large Language Model Agent Framework for Urban Knowledge Graph Construction

Published 10 Feb 2024 in cs.AI | (2402.06861v2)

Abstract: Urban knowledge graph has recently worked as an emerging building block to distill critical knowledge from multi-sourced urban data for diverse urban application scenarios. Despite its promising benefits, urban knowledge graph construction (UrbanKGC) still heavily relies on manual effort, hindering its potential advancement. This paper presents UrbanKGent, a unified LLM agent framework, for urban knowledge graph construction. Specifically, we first construct the knowledgeable instruction set for UrbanKGC tasks (such as relational triplet extraction and knowledge graph completion) via heterogeneity-aware and geospatial-infused instruction generation. Moreover, we propose a tool-augmented iterative trajectory refinement module to enhance and refine the trajectories distilled from GPT-4. Through hybrid instruction fine-tuning with augmented trajectories on Llama 2 and Llama 3 family, we obtain UrbanKGC agent family, consisting of UrbanKGent-7/8/13B version. We perform a comprehensive evaluation on two real-world datasets using both human and GPT-4 self-evaluation. The experimental results demonstrate that UrbanKGent family can not only significantly outperform 31 baselines in UrbanKGC tasks, but also surpass the state-of-the-art LLM, GPT-4, by more than 10% with approximately 20 times lower cost. Compared with the existing benchmark, the UrbanKGent family could help construct an UrbanKG with hundreds of times richer relationships using only one-fifth of the data. Our data and code are available at https://github.com/usail-hkust/UrbanKGent.

Authors (2)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (58)
  1. 2023. Significant-gravitas/auto-gpt: An experimental open-source attempt to make gpt-4 fully autonomous. (2023).
  2. Are Large Language Models Geospatially Knowledgeable?. In Proceedings of the 31st ACM International Conference on Advances in Geographic Information Systems. 1–4.
  3. Large language models as tool makers. arXiv preprint arXiv:2305.17126 (2023).
  4. Fireact: Toward language agent fine-tuning. arXiv preprint arXiv:2310.05915 (2023).
  5. Learning deep representation from big and heterogeneous data for traffic accident inference. In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, February 12-17, 2016, Phoenix, Arizona, USA. 338–344.
  6. Evaluating hallucinations in chinese large language models. arXiv preprint arXiv:2310.03368 (2023).
  7. A neural attention model for urban air quality inference: Learning the weights of monitoring stations. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, AAAI 2018, USA, February 2-7, 2018. 2151–2158.
  8. RelationPrompt: Leveraging prompts to generate synthetic data for zero-shot relation triplet extraction. arXiv preprint arXiv:2203.09101 (2022).
  9. Friendship and mobility: user movement in location-based social networks. In Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining. 1082–1090.
  10. Learning A Foundation Language Model for Geoscience Knowledge Understanding and Utilization. arXiv preprint arXiv:2306.05064 (2023).
  11. Agents: An Open-source Framework for Autonomous Language Agents. ArXiv abs/2309.07870 (2023). https://api.semanticscholar.org/CorpusID:261822166
  12. The Rise and Potential of Large Language Model Based Agents: A Survey. ArXiv abs/2309.07864 (2023). https://api.semanticscholar.org/CorpusID:261817592
  13. Mathematical capabilities of chatgpt. arXiv preprint arXiv:2301.13867 (2023).
  14. Gptscore: Evaluate as you desire. arXiv preprint arXiv:2302.04166 (2023).
  15. Specializing Smaller Language Models towards Multi-Step Reasoning. arXiv preprint arXiv:2301.12726 (2023).
  16. A real-world webagent with planning, long context understanding, and program synthesis. arXiv preprint arXiv:2307.12856 (2023).
  17. Faithful Question Answering with Monte-Carlo Planning. arXiv preprint arXiv:2305.02556 (2023).
  18. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685 (2021).
  19. Large language models cannot self-correct reasoning yet. arXiv preprint arXiv:2310.01798 (2023).
  20. Large Language Models as Traffic Signal Control Agents: Capacity and Opportunity. arXiv preprint arXiv:2312.16044 (2023).
  21. Camel: Communicative agents for” mind” exploration of large scale language model society. arXiv preprint arXiv:2303.17760 (2023).
  22. Revisiting Large Language Models as Zero-shot Relation Extractors. arXiv preprint arXiv:2310.05028 (2023).
  23. GeoLM: Empowering Language Models for Geospatially Grounded Language Understanding. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 5227–5240.
  24. GeoGalactica: A Scientific Large Language Model in Geoscience. arXiv preprint arXiv:2401.00434 (2023).
  25. Urban flow pattern mining based on multi-source heterogeneous data fusion and knowledge graph embedding. IEEE Transactions on Knowledge and Data Engineering (2021).
  26. UrbanKG: An Urban Knowledge Graph System. ACM Transactions on Intelligent Systems and Technology (2023), 1–25.
  27. Developing knowledge graph based system for urban computing. In Proceedings of the 1st ACM SIGSPATIAL International Workshop on Geospatial Knowledge Graphs. 3–7.
  28. PIVOINE: Instruction Tuning for Open-world Information Extraction. arXiv preprint arXiv:2305.14898 (2023).
  29. Self-refine: Iterative refinement with self-feedback. arXiv preprint arXiv:2303.17651 (2023).
  30. SpatialML: annotation scheme, resources, and evaluation. Language Resources and Evaluation (2010), 263–280.
  31. Geollm: Extracting geospatial knowledge from large language models. arXiv preprint arXiv:2310.06213 (2023).
  32. Towards Understanding the Geospatial Skills of ChatGPT: Taking a Geographic Information Systems (GIS) Exam. In Proceedings of the 6th ACM SIGSPATIAL International Workshop on AI for Geographic Knowledge Discovery. 85–94.
  33. UUKG: Unified Urban Knowledge Graph Dataset for Urban Spatiotemporal Prediction. arXiv preprint arXiv:2306.11443 (2023).
  34. Sequence-to-sequence knowledge graph completion and question answering. arXiv preprint arXiv:2203.10321 (2022).
  35. DxNAT—Deep neural networks for explaining non-recurring traffic congestion. In 2017 IEEE International Conference on Big Data (IEEE BigData 2017), Boston, MA, USA, December 11-14, 2017. 2141–2150.
  36. Research on the construction of a knowledge graph and knowledge reasoning model in the field of urban traffic. Sustainability (2021), 3191.
  37. Llama 2: Open Foundation and Fine-Tuned Chat Models. CoRR abs/2307.09288 (2023). arXiv:2307.09288
  38. Evaluating open question answering evaluation. arXiv preprint arXiv:2305.12421 (2023).
  39. Voyager: An open-ended embodied agent with large language models. arXiv preprint arXiv:2305.16291 (2023).
  40. Spatio-temporal urban knowledge graph enabled mobility prediction. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., 184:1–184:24.
  41. Knowledge Graph of Urban Firefighting with Rule-Based Entity Extraction. In International Conference on Engineering Applications of Neural Networks. Springer, 168–177.
  42. Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, 24824–24837.
  43. Zero-shot information extraction via chatting with chatgpt. arXiv preprint arXiv:2302.10205 (2023).
  44. Empirical study of zero-shot ner with chatgpt. arXiv preprint arXiv:2310.10035 (2023).
  45. Chengbiao Yang and Guilin Qi. 2022. An urban traffic knowledge graph-driven spatial-temporal graph convolutional network for traffic flow prediction. In Proceedings of the 11th International Joint Conference on Knowledge Graphs. 110–114.
  46. KG-BERT: BERT for knowledge graph completion. arXiv preprint arXiv:1909.03193 (2019).
  47. Exploring large language models for knowledge graph completion. arXiv preprint arXiv:2308.13916 (2023).
  48. Generative knowledge graph construction: A review. arXiv preprint arXiv:2210.12714 (2022).
  49. Zero-shot temporal relation extraction with chatgpt. arXiv preprint arXiv:2304.05454 (2023).
  50. Jerrold H Zar. 2005. Spearman rank correlation. Encyclopedia of biostatistics (2005).
  51. Agenttuning: Enabling generalized agent abilities for llms. arXiv preprint arXiv:2310.12823 (2023).
  52. Yueling Zeng and Li-C Wang. 2023. Domain Knowledge Graph Construction Via A Simple Checker. arXiv preprint arXiv:2310.04949 (2023).
  53. Evaluating and Improving Tool-Augmented Computation-Intensive Math Reasoning. arXiv preprint arXiv:2306.02408 (2023).
  54. Deep transfer learning for city-scale cellular traffic generation through urban knowledge graph. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 4842–4851.
  55. Urban multi-source spatio-temporal data analysis aware knowledge graph embedding. Symmetry (2020), 199.
  56. PRGC: Potential relation and global correspondence based joint relational triple extraction. arXiv preprint arXiv:2106.09895 (2021).
  57. Judging LLM-as-a-judge with MT-Bench and Chatbot Arena. arXiv preprint arXiv:2306.05685 (2023).
  58. Hierarchical knowledge graph learning enabled socioeconomic indicator prediction in location-based social network. In Proceedings of the ACM Web Conference. 122–132.
Citations (1)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up

Tweets

Sign up for free to view the 1 tweet with 1 like about this paper.

Sign Up for Free