Papers
Topics
Authors
Recent
Detailed Answer
Quick Answer
Concise responses based on abstracts only
Detailed Answer
Well-researched responses based on abstracts and relevant paper content.
Custom Instructions Pro
Preferences or requirements that you'd like Emergent Mind to consider when generating responses
Gemini 2.5 Flash
Gemini 2.5 Flash 82 tok/s
Gemini 2.5 Pro 47 tok/s Pro
GPT-5 Medium 14 tok/s Pro
GPT-5 High 16 tok/s Pro
GPT-4o 117 tok/s Pro
Kimi K2 200 tok/s Pro
GPT OSS 120B 469 tok/s Pro
Claude Sonnet 4 36 tok/s Pro
2000 character limit reached

PlanGPT: Enhancing Urban Planning with Tailored Language Model and Efficient Retrieval (2402.19273v1)

Published 29 Feb 2024 in cs.CL

Abstract: In the field of urban planning, general-purpose LLMs often struggle to meet the specific needs of planners. Tasks like generating urban planning texts, retrieving related information, and evaluating planning documents pose unique challenges. To enhance the efficiency of urban professionals and overcome these obstacles, we introduce PlanGPT, the first specialized LLM tailored for urban and spatial planning. Developed through collaborative efforts with institutions like the Chinese Academy of Urban Planning, PlanGPT leverages a customized local database retrieval framework, domain-specific fine-tuning of base models, and advanced tooling capabilities. Empirical tests demonstrate that PlanGPT has achieved advanced performance, delivering responses of superior quality precisely tailored to the intricacies of urban planning.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (83)
  1. Deepspeed-inference: enabling efficient inference of transformer models at unprecedented scale. In SC22: International Conference for High Performance Computing, Networking, Storage and Analysis, pages 1–15. IEEE.
  2. Anthropic. 2023. Model card and evaluations for claude models.
  3. Self-rag: Learning to retrieve, generate, and critique through self-reflection. arXiv preprint arXiv:2310.11511.
  4. Baichuan. 2023. Baichuan 2: Open large-scale language models. arXiv preprint arXiv:2309.10305.
  5. Improving language models by retrieving from trillions of tokens. In International conference on machine learning, pages 2206–2240. PMLR.
  6. A large annotated corpus for learning natural language inference. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics.
  7. SemEval-2017 task 1: Semantic textual similarity multilingual and crosslingual focused evaluation. In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), pages 1–14, Vancouver, Canada. Association for Computational Linguistics.
  8. Bge m3-embedding: Multi-lingual, multi-functionality, multi-granularity text embeddings through self-knowledge distillation.
  9. Lift yourself up: Retrieval-augmented text generation with self-memory. Advances in Neural Information Processing Systems, 36.
  10. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality.
  11. Deep reinforcement learning from human preferences.
  12. Chatlaw. https://github.com/PKU-YuanGroup/ChatLaw.
  13. Pre-training with whole word masking for chinese bert.
  14. Lert: A linguistically-motivated pre-trained language model.
  15. Efficient and effective text encoding for chinese llama and alpaca. arXiv preprint arXiv:2304.08177.
  16. Tri Dao. 2023. Flashattention-2: Faster attention with better parallelism and work partitioning. arXiv preprint arXiv:2307.08691.
  17. Google DeepMind. 2023a. Bard. https://bard.google.com.
  18. Google DeepMind. 2023b. Gemini. https://gemini.google.com.
  19. Mods: Model-oriented data selection for instruction tuning. arXiv preprint arXiv:2311.15653.
  20. Glm: General language model pretraining with autoregressive blank infilling. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 320–335.
  21. Jakubik et al. 2023a. Prithvi-100M.
  22. Rohan Anil et al. 2023b. Palm 2 technical report.
  23. A framework for few-shot language model evaluation.
  24. Simcse: Simple contrastive learning of sentence embeddings.
  25. HIT-SCIR. 2024. Chinese-mixtral-8x7b: An open-source mixture-of-experts llm. https://github.com/HIT-SCIR/Chinese-Mixtral-8x7B.
  26. Metagpt: Meta programming for multi-agent collaborative framework. arXiv preprint arXiv:2308.00352.
  27. Raven: In-context learning with retrieval augmented encoder-decoder language models. arXiv preprint arXiv:2308.07922.
  28. C-eval: A multi-level multi-discipline chinese evaluation suite for foundation models. In Advances in Neural Information Processing Systems.
  29. Active retrieval augmented generation. arXiv preprint arXiv:2305.06983.
  30. Tree of clarifications: Answering ambiguous questions with retrieval-augmented large language models. arXiv preprint arXiv:2310.14696.
  31. Efficient memory management for large language model serving with pagedattention.
  32. Cmmlu: Measuring massive multitask language understanding in chinese.
  33. From quantity to quality: Boosting llm performance with self-guided data selection for instruction tuning. ArXiv, abs/2308.12032.
  34. Self-alignment with instruction backtranslation. arXiv preprint arXiv:2308.06259.
  35. Ra-dit: Retrieval-augmented dual instruction tuning. arXiv preprint arXiv:2310.01352.
  36. C. Liu and W. Zhang. 2023. Social and spatial heterogeneities in covid-19 impacts on individual’s metro use: A big-data driven causality inference. Applied Geography, 155:102947.
  37. What makes good data for alignment? a comprehensive study of automatic data selection in instruction tuning. In The Twelfth International Conference on Learning Representations.
  38. Webglm: Towards an efficient web-enhanced question answering system with human preferences.
  39. LCQMC:a large-scale Chinese question matching corpus. In Proceedings of the 27th International Conference on Computational Linguistics, pages 1952–1962, Santa Fe, New Mexico, USA. Association for Computational Linguistics.
  40. Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization.
  41. Muffin: Curating multi-faceted instructions for improving instruction following. In The Twelfth International Conference on Learning Representations.
  42. Mobilityagent. https://github.com/XiaoLeGG/mobility-agent.
  43. Efficient estimation of word representations in vector space.
  44. Mistral-AI. 2023. mistral. https://mistral.ai/.
  45. Yohei Nakajima. Babyagi, 2023. URL https://github. com/yoheinakajima/babyagi. GitHub repository.
  46. Webgpt: Browser-assisted question-answering with human feedback. arXiv preprint arXiv:2112.09332.
  47. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748.
  48. OpenAI. 2022. Chatgpt. https://chat.openai.com.
  49. OpenAI. 2023. Gpt-4 technical report.
  50. Wang Peng. 2023. Duomo/transgpt.
  51. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36.
  52. Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks.
  53. Ozan Sener and Silvio Savarese. 2017. Active learning for convolutional neural networks: A core-set approach. arXiv preprint arXiv:1708.00489.
  54. Threshold and moderating effects of land use on metro ridership in shenzhen: Implications for tod planning. Journal of Transport Geography, 89:102878.
  55. Built environment interventions for emission mitigation: A machine learning analysis of travel-related co2 in a developing city. Journal of Transport Geography, 110:103632.
  56. Significant Gravitas. AutoGPT.
  57. Charles Spearman. 1961. The proof and measurement of association between two things.
  58. Jianlin Su. 2022. Cosent.
  59. Recitation-augmented language models. arXiv preprint arXiv:2210.01296.
  60. Stanford alpaca: An instruction-following llama model. https://github.com/tatsu-lab/stanford_alpaca.
  61. Lagent Developer Team. 2023a. Lagent: InternLM a lightweight open-source framework that allows users to efficiently build large language model(llm)-based agents. https://github.com/InternLM/lagent.
  62. XAgent Team. 2023b. Xagent: An autonomous agent for complex task solving.
  63. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971.
  64. Krish Mangroila Tycho Young, Andy Zhang. 2023. Mathgpt - an exploration into the field of mathematics with large language models.
  65. Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-sne. Journal of machine learning research, 9(11).
  66. Huatuo: Tuning llama model with chinese medical knowledge. arXiv preprint arXiv:2304.06975.
  67. Self-instruct: Aligning language model with self generated instructions. arXiv preprint arXiv:2212.10560.
  68. Skywork: A more open bilingual foundation model.
  69. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38–45, Online. Association for Computational Linguistics.
  70. Autogen: Enabling next-gen llm applications via multi-agent conversation framework.
  71. Openagents: An open platform for language agents in the wild. arXiv preprint arXiv:2310.10634.
  72. Doctorglm: Fine-tuning your chinese doctor is not a herculean task. arXiv preprint arXiv:2304.01097.
  73. Wizardlm: Empowering large language models to follow complex instructions. arXiv preprint arXiv:2304.12244.
  74. PAWS-X: A cross-lingual adversarial dataset for paraphrase identification. CoRR, abs/1908.11828.
  75. Generate rather than retrieve: Large language models are strong context generators. arXiv preprint arXiv:2209.10063.
  76. Trafficgpt: Viewing, processing and interacting with traffic foundation models.
  77. Measuring megaregional structure in the pearl river delta by mobile phone signaling data: A complex network approach. Cities, 104:102809.
  78. Incorporating polycentric development and neighborhood life-circle planning for reducing driving in beijing: Nonlinear and threshold analysis. Cities, 121:103488.
  79. W. Zhang and K. Ning. 2023. Spatiotemporal heterogeneities in the causal effects of mobility intervention policies during the covid-19 outbreak: A spatially interrupted time-series (sits) analysis. Annals of the American Association of Geographers, 113(5):1112–1134.
  80. Earthgpt: A universal multi-modal large language model for multi-sensor image comprehension in remote sensing domain. arXiv preprint arXiv:2401.16822.
  81. Xuanyuan 2.0: A large chinese financial chat model with hundreds of billions parameters.
  82. Kun: Answer polishment for chinese self-alignment with instruction back-translation. arXiv preprint arXiv:2401.06477.
  83. Lima: Less is more for alignment. Advances in Neural Information Processing Systems, 36.
Citations (3)
View on Semantic Scholar
List To Do Tasks Checklist Streamline Icon: https://streamlinehq.com

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up

Summary

  • The paper demonstrates PlanGPT’s ability to outperform general LLMs by accurately generating urban planning texts and retrieving relevant data.
  • It details a customized retrieval framework and domain-specific fine-tuning that enhance understanding of planning documents.
  • The research highlights PlanGPT’s potential to streamline document evaluation and drive more efficient urban planning practices.

Enhancing Urban Planning Through PlanGPT: A Specialized LLM

Introduction to PlanGPT

The field of urban planning presents distinct challenges that general-purpose LLMs often struggle to address effectively. Urban planning tasks require the generation of specific texts, the retrieval of related information, and a thorough evaluation of planning documents, all within the unique terminologies and structures of urban planning discourse. To meet these needs, PlanGPT emerges as the first specialized LLM tailored for the domain of urban and spatial planning. Its development involved a collaboration with esteemed institutions, including the Chinese Academy of Urban Planning, aiming to boost the productivity and efficiency of urban planning professionals.

Key Features and Innovations of PlanGPT

PlanGPT distinguishes itself through several innovative features designed to address the core challenges in urban planning:

  • The model employs a customized local database retrieval framework to efficiently handle vast amounts of urban planning texts, overcoming the low signal-to-noise ratio characteristic of such documents.
  • It incorporates domain-specific fine-tuning techniques that enhance its proficiency in understanding and generating government-style documents.
  • PlanGPT also utilizes advanced tooling through PlanAgent, leveraging data resources (such as networks, visual aids, and charts) to handle the timeliness and multimodality necessary for comprehending urban planning documents.

PlanGPT’s Empirical Performance

Empirical tests have showcased PlanGPT’s superior performance in typical urban planning tasks, outperforming state-of-the-art models in generating urban planning texts, information retrieval, and document evaluation. This illustrates PlanGPT's capability to produce responses precisely tailored to the intricacies of urban planning, embodying both theoretical and practical advancements in the application of LLMs within this specialized field.

The exploration of LLMs, both general-purpose and vertical-specific, reveals a broad array of applications across different sectors. Prior models tailored to fields such as finance, healthcare, and law have shown the potential of specialized LLMs in addressing domain-specific challenges. However, the urban planning sector remains largely untapped, with existing models lacking the capability to fully engage with the specialized knowledge and document styles of urban planning. PlanGPT, therefore, represents a pioneering effort to fill this gap by offering a solution explicitly designed for urban planning purposes.

Theoretical and Practical Implications

PlanGPT sets a precedent for the creation of domain-specific LLMs, particularly in disciplines that involve complex interdisciplinary knowledge and specialized document styles. The model’s capacity for understanding and generating urban planning texts augments theoretical discussions on the flexibility and adaptability of LLMs to different domains. Practically, PlanGPT promises to enhance the productivity of urban planning professionals by automating tasks that traditionally require extensive time and effort, such as document reviews and assessments.

Future Prospects in AI and Urban Planning

Looking forward, the ongoing development and refinement of PlanGPT will likely focus on integrating more holistic multimodal data processing capabilities and expanding its knowledge base within the urban planning domain. This direction not only underscores the model’s potential to evolve in response to emerging urban planning challenges but also hints at broader implications for the incorporation of AI technologies in enhancing urban development processes and policies.

Conclusion

PlanGPT signifies a significant step towards integrating AI with urban and spatial planning, offering a specialized tool that addresses specific challenges in this field. By leveraging advanced LLM capabilities tailored for urban planning, PlanGPT showcases the potential of domain-specific models to enhance the effectiveness and efficiency of urban professionals. As this body of work progresses, it is anticipated that further advancements in AI will continue to reshape the landscape of urban planning, driving innovations that support sustainable and intelligent urban development.

File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown
Dice Question Streamline Icon: https://streamlinehq.com

Follow-Up Questions

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now