Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
102 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

AIOS Compiler: LLM as Interpreter for Natural Language Programming and Flow Programming of AI Agents (2405.06907v2)

Published 11 May 2024 in cs.CL, cs.AI, cs.LG, and cs.PL

Abstract: Since their inception, programming languages have trended towards greater readability and lower barriers for programmers. Following this trend, natural language can be a promising type of programming language that provides great flexibility and usability and helps towards the democracy of programming. However, the inherent vagueness, ambiguity, and verbosity of natural language pose significant challenges in developing an interpreter that can accurately understand the programming logic and execute instructions written in natural language. Fortunately, recent advancements in LLMs have demonstrated remarkable proficiency in interpreting complex natural language. Inspired by this, we develop a novel system for Code Representation and Execution (CoRE), which employs LLM as interpreter to interpret and execute natural language instructions. The proposed system unifies natural language programming, pseudo-code programming, and flow programming under the same representation for constructing language agents, while LLM serves as the interpreter to interpret and execute the agent programs. In this paper, we begin with defining the programming syntax that structures natural language instructions logically. During the execution, we incorporate external memory to minimize redundancy. Furthermore, we equip the designed interpreter with the capability to invoke external tools, compensating for the limitations of LLM in specialized domains or when accessing real-time information. This work is open-source at https://github.com/agiresearch/CoRE, https://github.com/agiresearch/OpenAGI, and https://github.com/agiresearch/AIOS.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (53)
  1. Do as i can, not as i say: Grounding language in robotic affordances. arXiv preprint arXiv:2204.01691 (2022).
  2. Graph of thoughts: Solving elaborate problems with large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 38. 17682–17690.
  3. Improving language models by retrieving from trillions of tokens. In International conference on machine learning. PMLR, 2206–2240.
  4. Amy Bruckman and Elizabeth Edwards. 1999. Should we leverage natural-language knowledge? An analysis of user errors in a natural-language-style programming language. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems. 207–214.
  5. Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks. Transactions on Machine Learning Research (2023).
  6. Teaching large language models to self-debug. arXiv preprint arXiv:2304.05128 (2023).
  7. Scaling instruction-finetuned language models. Journal of Machine Learning Research 25, 70 (2024), 1–53.
  8. Structured programming. Academic Press Ltd.
  9. Program synthesis using natural language. In Proceedings of the 38th International Conference on Software Engineering. 345–356.
  10. Task and motion planning with large language models for object rearrangement. In 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2086–2092.
  11. Palm-e: An embodied multimodal language model. arXiv preprint arXiv:2303.03378 (2023).
  12. Michael D Ernst. 2017. Natural language is a programming language: Applying natural language processing to software development. In 2nd Summit on Advances in Programming Languages (SNAPL 2017). Schloss-Dagstuhl-Leibniz Zentrum für Informatik.
  13. OpenAGI: When LLM Meets Domain Experts. In Advances in Neural Information Processing Systems (NeurIPS) (2023).
  14. LLM as OS, Agents as Apps: Envisioning AIOS, Agents and the AIOS-Agent Ecosystem. arXiv e-prints (2023), arXiv–2312.
  15. Reasoning with language model is planning with world model. arXiv preprint arXiv:2305.14992 (2023).
  16. George E Heidorn. 1976. Automatic programming through natural language dialogue: A survey. IBM Journal of research and development 20, 4 (1976), 302–313.
  17. John L Hennessy and David A Patterson. 2011. Computer architecture: a quantitative approach. Elsevier.
  18. CLIPScore: A Reference-free Evaluation Metric for Image Captioning.
  19. Inner monologue: Embodied reasoning through planning with language models. arXiv preprint arXiv:2207.05608 (2022).
  20. Mixtral of experts. arXiv preprint arXiv:2401.04088 (2024).
  21. Gpt is becoming a turing machine: Here are some ways to program it. arXiv preprint arXiv:2303.14310 (2023).
  22. Demonstrate-Search-Predict: Composing Retrieval and Language Models for Knowledge-Intensive NLP. arXiv preprint arXiv:2212.14024 (2022).
  23. DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines. arXiv preprint arXiv:2310.03714 (2023).
  24. Large language models are zero-shot reasoners. Advances in neural information processing systems 35 (2022), 22199–22213.
  25. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in Neural Information Processing Systems 33 (2020), 9459–9474.
  26. Jiwei Li and Eduard Hovy. 2015. The NLP engine: A universal turing machine for nlp. arXiv preprint arXiv:1503.00168 (2015).
  27. Formal-LLM: Integrating Formal Language and Natural Language for Controllable LLM-based Agents. arXiv:2402.00798 (2024).
  28. Taskmatrix. ai: Completing tasks by connecting foundation models with millions of apis. arXiv preprint arXiv:2303.16434 (2023).
  29. Llm+ p: Empowering large language models with optimal planning proficiency. arXiv preprint arXiv:2304.11477 (2023).
  30. Faithful chain-of-thought reasoning. arXiv preprint arXiv:2301.13379 (2023).
  31. Self-refine: Iterative refinement with self-feedback. Advances in Neural Information Processing Systems 36 (2024).
  32. AIOS: LLM Agent Operating System. arXiv (2024).
  33. NLP (natural language processing) for NLP (natural language programming). In Computational Linguistics and Intelligent Text Processing: 7th International Conference, CICLing 2006, Mexico City, Mexico, February 19-25, 2006. Proceedings 7. Springer, 319–330.
  34. Codegen: An open large language model for code with multi-turn program synthesis. arXiv preprint arXiv:2203.13474 (2022).
  35. Josh et al OpenAI. 2023. GPT-4 Technical Report. arXiv:2303.08774 [cs.CL]
  36. Training language models to follow instructions with human feedback. Advances in neural information processing systems 35 (2022), 27730–27744.
  37. Refiner: Reasoning feedback on intermediate representations. arXiv preprint arXiv:2304.01904 (2023).
  38. Synchromesh: Reliable code generation from pre-trained language models. arXiv preprint arXiv:2201.11227 (2022).
  39. Ronald E Prather. 1997. Regular expressions for program computations. The American mathematical monthly 104, 2 (1997), 120–130.
  40. Toolllm: Facilitating large language models to master 16000+ real-world apis. arXiv preprint arXiv:2307.16789 (2023).
  41. The programmer’s assistant: Conversational interaction with a large language model for software development. In Proceedings of the 28th International Conference on Intelligent User Interfaces. 491–514.
  42. Reflexion: an autonomous agent with dynamic memory and self-reflection. arXiv preprint arXiv:2303.11366 (2023).
  43. Progprompt: Generating situated robot task plans using large language models. In 2023 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 11523–11530.
  44. David Vadas and James R Curran. 2005. Programming with unrestricted natural language. In Proceedings of the Australasian Language Technology Workshop 2005. 191–199.
  45. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171 (2022).
  46. Augmenting black-box llms with medical textbooks for clinical question answering. arXiv preprint arXiv:2309.02233 (2023).
  47. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems 35 (2022), 24824–24837.
  48. Visual Transformers: Token-based Image Representation and Processing for Computer Vision. arXiv:2006.03677 [cs.CV]
  49. StateFlow: Enhancing LLM Task-Solving through State-Driven Workflows. arXiv preprint arXiv:2403.11322 (2024).
  50. Tree of thoughts: Deliberate problem solving with large language models. Advances in Neural Information Processing Systems 36 (2024).
  51. React: Synergizing reasoning and acting in language models. arXiv preprint arXiv:2210.03629 (2022).
  52. ReAct: Synergizing Reasoning and Acting in Language Models. In International Conference on Learning Representations (ICLR).
  53. BERTScore: Evaluating Text Generation with BERT.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (4)
  1. Shuyuan Xu (31 papers)
  2. Zelong Li (24 papers)
  3. Kai Mei (30 papers)
  4. Yongfeng Zhang (163 papers)
Citations (2)
View on Semantic Scholar
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets

HackerNews

  1. LLM as Interpreter for Natural Language Programming (2 points, 0 comments)