Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
144 tokens/sec
GPT-4o
8 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

LangGPT: Rethinking Structured Reusable Prompt Design Framework for LLMs from the Programming Language (2402.16929v2)

Published 26 Feb 2024 in cs.SE, cs.AI, cs.CL, and cs.PL

Abstract: LLMs have demonstrated commendable performance across diverse domains. Nevertheless, formulating high-quality prompts to instruct LLMs proficiently poses a challenge for non-AI experts. Existing research in prompt engineering suggests somewhat scattered optimization principles and designs empirically dependent prompt optimizers. Unfortunately, these endeavors lack a structured design template, incurring high learning costs and resulting in low reusability. In addition, it is not conducive to the iterative updating of prompts. Inspired by structured reusable programming languages, we propose LangGPT, a dual-layer prompt design framework as the programming language for LLMs. LangGPT has an easy-to-learn normative structure and provides an extended structure for migration and reuse. Experiments illustrate that LangGPT significantly enhances the performance of LLMs. Moreover, the case study shows that LangGPT leads LLMs to generate higher-quality responses. Furthermore, we analyzed the ease of use and reusability of LangGPT through a user survey in our online community.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (58)
  1. 01-ai. Building the Next Generation of Open-Source and Bilingual LLMs. https://github.com/01-ai/Yi, 2023. original-date: 2023-11-03T16:08:37Z.
  2. GPT-4 Technical Report, December 2023. arXiv:2303.08774 [cs].
  3. Alfred V Aho. Compilers: principles, techniques and tools. Pearson Education India, 2007.
  4. Nabil Alouani. Prompt Engineering Could Be the Hottest Programming Language of 2024 — Here’s Why. https://towardsdatascience.com, December 2023.
  5. Gemini: A Family of Highly Capable Multimodal Models, December 2023. arXiv:2312.11805 [cs].
  6. Qwen Technical Report, September 2023. arXiv:2309.16609 [cs].
  7. Principled Instructions Are All You Need for Questioning LLaMA-1/2, GPT-3.5/4, December 2023. arXiv:2312.16171 [cs].
  8. A study on Prompt Design, Advantages and Limitations of ChatGPT for Deep Learning Program Repair, April 2023. arXiv:2304.08191 [cs].
  9. Chakray. Programming Languages: Types and Features, December 2018.
  10. Harrison Chase. LangChain: Building applications with LLMs through composability. https://github.com/langchain-ai/langchain, October 2022.
  11. Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review, October 2023. arXiv:2310.14735 [cs] version: 2.
  12. Black-Box Prompt Optimization: Aligning Large Language Models without Model Training, November 2023. arXiv:2311.04155 [cs].
  13. GLM: General Language Model Pretraining with Autoregressive Blank Infilling. In Annual Meeting of the Association for Computational Linguistics, 2021.
  14. Mihail Eric. A complete introduction to prompt engineering for large language models. https://www.mihaileric.com, October 2022.
  15. Towards an Automatic Prompt Optimization Framework for AI Image Generation. In HCI International 2023 Posters, Communications in Computer and Information Science, pages 405–410, Cham, 2023. Springer Nature Switzerland.
  16. An Introduction to Language. Cengage Learning, 2018.
  17. Nagaraju Gajula. A Guide to Prompt Engineering in Large Language Models. https://www.latentview.com, August 2023.
  18. GeeksforGeeks. Natural Language Processing(NLP) VS Programming Language. https://www.geeksforgeeks.org, December 2023. Section: Python.
  19. John Gruber. Markdown: Syntax. http://daringfireball.net/projects/markdown, 2012.
  20. Modern compiler design. Springer Science & Business Media, 2012.
  21. Connecting Large Language Models with Evolutionary Algorithms Yields Powerful Prompt Optimizers, September 2023. arXiv:2309.08532 [cs].
  22. Optimizing Prompts for Text-to-Image Generation, December 2022. arXiv:2212.09611 [cs].
  23. MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework, November 2023. arXiv:2308.00352 [cs].
  24. Dialogue for Prompting: a Policy-Gradient-Based Discrete Prompt Optimization for Few-shot Learning, August 2023. arXiv:2308.07272 [cs].
  25. Chatdoctor: A medical chat model fine-tuned on a large language model meta-ai (llama) using medical domain knowledge. Cureus, 15(6), 2023.
  26. Design Guidelines for Prompt Engineering Text-to-Image Generative Models. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, CHI ’22, pages 1–23, New York, NY, USA, April 2022. Association for Computing Machinery.
  27. Calibrating LLM-Based Evaluator, September 2023. arXiv:2309.13308 [cs].
  28. Mark Lutz. Programming Python: Powerful Object-Oriented Programming. ”O’Reilly Media, Inc.”, December 2010.
  29. Bertalan Meskó. Prompt Engineering as an Important Emerging Skill for Medical Professionals: Tutorial. Journal of Medical Internet Research, 25(1):e50638, October 2023.
  30. Shritam Kumar Mund. The AI War: Mastering the Art of Prompt Engineering in the Era of Large Language Models. https://ai.plainenglish.io, February 2023.
  31. Matt Nigh. ChatGPT3-Free-Prompt-List: A free guide for learning to create ChatGPT3 Prompts. https://github.com/mattnigh/ChatGPT3-Free-Prompt-List, 2023.
  32. Generative Agents: Interactive Simulacra of Human Behavior. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology, UIST ’23, pages 1–22, New York, NY, USA, October 2023. Association for Computing Machinery.
  33. Foundations of JSON Schema. In Proceedings of the 25th International Conference on World Wide Web, WWW ’16, pages 263–273, Republic and Canton of Geneva, CHE, April 2016. International World Wide Web Conferences Steering Committee.
  34. Programming languages: design and implementation. Technical report, Prentice-Hall Englewood Cliffs, NJ, 1984.
  35. Automatic Prompt Optimization with ”Gradient Descent” and Beam Search, October 2023. arXiv:2305.03495 [cs].
  36. Techgpt 2.0: Technology-oriented generative pretrained transformer 2.0. https://github.com/neukg/TechGPT-2.0, 2023.
  37. Tim Rentsch. Object oriented programming. ACM SIGPLAN Notices, 17(9):51–57, September 1982.
  38. Introducing ChatGPT. https://openai.com/blog/chatgpt, 2023.
  39. Robert W Sebesta. Concepts of programming languages. Pearson Education, Inc, 2012.
  40. Michael Sipser. Introduction to the Theory of Computation. ACM Sigact News, 27(1):27–29, 1996. Publisher: ACM New York, NY, USA.
  41. ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generation, July 2021. arXiv:2107.02137 [cs].
  42. AutoHint: Automatic Prompt Optimization with Hint Generation, August 2023. arXiv:2307.07415 [cs].
  43. A Survey of Reasoning with Foundation Models, December 2023. arXiv:2312.11562 [cs].
  44. Evaluating Large Language Models on Controlled Generation Tasks, October 2023. arXiv:2310.14542 [cs].
  45. An Introduction to Large Language Models: Prompt Engineering and P-Tuning. https://developer.nvidia.com, April 2023.
  46. HuaTuo: Tuning LLaMA Model with Chinese Medical Knowledge, April 2023. arXiv:2304.06975 [cs].
  47. PromptAgent: Strategic Planning with Language Models Enables Expert-level Prompt Optimization, December 2023. arXiv:2310.16427 [cs].
  48. Zhenxuan Wang. How to use prompt engineering with large language models. https://www.thoughtworks.com, August 2023.
  49. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models, January 2023. arXiv:2201.11903 [cs].
  50. AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation, October 2023. arXiv:2308.08155 [cs].
  51. The Rise and Potential of Large Language Model Based Agents: A Survey, September 2023. arXiv:2309.07864 [cs].
  52. Exploring Large Language Models for Communication Games: An Empirical Study on Werewolf, September 2023. arXiv:2309.04658 [cs].
  53. Baichuan 2: Open Large-scale Language Models, September 2023. arXiv:2309.10305 [cs].
  54. Decorate the Examples: A Simple Method of Prompt Design for Biomedical Relation Extraction, April 2022. arXiv:2204.10360 [cs].
  55. Large Language Model as Attributed Training Data Generator: A Tale of Diversity and Bias, October 2023. arXiv:2306.15895 [cs].
  56. Why Johnny Can’t Prompt: How Non-AI Experts Try (and Fail) to Design LLM Prompts. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, CHI ’23, pages 1–21, New York, NY, USA, April 2023. Association for Computing Machinery.
  57. GLM-130B: An Open Bilingual Pre-trained Model, October 2023. arXiv:2210.02414 [cs].
  58. PICA: Unleashing The Emotional Power of Large Language Model. https://github.com/NEU-DataMining/PICA, July 2023.
Citations (3)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets

Youtube Logo Streamline Icon: https://streamlinehq.com

YouTube