Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
173 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

LLM-Based Multi-Agent Systems for Software Engineering: Literature Review, Vision and the Road Ahead (2404.04834v3)

Published 7 Apr 2024 in cs.SE

Abstract: Integrating LLMs into autonomous agents marks a significant shift in the research landscape by offering cognitive abilities that are competitive with human planning and reasoning. This paper explores the transformative potential of integrating LLMs into Multi-Agent (LMA) systems for addressing complex challenges in software engineering (SE). By leveraging the collaborative and specialized abilities of multiple agents, LMA systems enable autonomous problem-solving, improve robustness, and provide scalable solutions for managing the complexity of real-world software projects. In this paper, we conduct a systematic review of recent primary studies to map the current landscape of LMA applications across various stages of the software development lifecycle (SDLC). To illustrate current capabilities and limitations, we perform two case studies to demonstrate the effectiveness of state-of-the-art LMA frameworks. Additionally, we identify critical research gaps and propose a comprehensive research agenda focused on enhancing individual agent capabilities and optimizing agent synergy. Our work outlines a forward-looking vision for developing fully autonomous, scalable, and trustworthy LMA systems, laying the foundation for the evolution of Software Engineering 2.0.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (49)
  1. Llm-deliberation: Evaluating llms with interactive multi-agent negotiation games. arXiv preprint arXiv:2309.17234 (2023).
  2. Agile software development methods: Review and analysis. arXiv preprint arXiv:1709.08439 (2017).
  3. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 (2023).
  4. Saaket Agashe. 2023. LLM-Coordination: Developing Coordinating Agents with Large Language Models. University of California, Santa Cruz.
  5. Stefano V Albrecht and Peter Stone. 2018. Autonomous agents modelling other agents: A comprehensive survey and open problems. Artificial Intelligence 258 (2018), 66–95.
  6. R Meredith Belbin and Victoria Brown. 2022. Team roles at work. Routledge.
  7. Harrison Chase. 2022. LangChain. https://github.com/langchain-ai/langchain
  8. Scalable multi-robot collaboration with large language models: Centralized or decentralized systems? arXiv preprint arXiv:2309.15943 (2023).
  9. Exploring Large Language Model based Intelligent Agents: Definitions, Methods, and Prospects. arXiv preprint arXiv:2401.03428 (2024).
  10. Self-collaboration Code Generation via ChatGPT. arXiv preprint arXiv:2304.07590 (2023).
  11. Improving factuality and reasoning in language models through multiagent debate. arXiv preprint arXiv:2305.14325 (2023).
  12. Cynthia Dwork. 2006. Differential privacy. In International colloquium on automata, languages, and programming. Springer, 1–12.
  13. Stan Franklin and Art Graesser. 1996. Is it an Agent, or just a Program?: A Taxonomy for Autonomous Agents. In International workshop on agent theories, architectures, and languages. Springer, 21–35.
  14. Joseph A Goguen and Charlotte Linde. 1993. Techniques for requirements elicitation. In [1993] Proceedings of the IEEE International Symposium on Requirements Engineering. IEEE, 152–164.
  15. Oded Goldreich. 1998. Secure multi-party computation. Manuscript. Preliminary version 78, 110 (1998), 1–108.
  16. LLM Multi-Agent Systems: Challenges and Open Problems. arXiv preprint arXiv:2402.03578 (2024).
  17. Ptm4tag: sharpening tag recommendation of stack overflow posts with pre-trained models. In Proceedings of the 30th IEEE/ACM International Conference on Program Comprehension. 1–11.
  18. A Universal Modular ACTOR Formalism for Artificial Intelligence. In Proceedings of the 3rd International Joint Conference on Artificial Intelligence. Standford, CA, USA, August 20-23, 1973, Nils J. Nilsson (Ed.). William Kaufmann, 235–245. http://ijcai.org/Proceedings/73/Papers/027B.pdf
  19. Metagpt: Meta programming for multi-agent collaborative framework. arXiv preprint arXiv:2308.00352 (2023).
  20. John J Horton. 2023. Large language models as simulated economic agents: What can we learn from homo silicus? Technical Report. National Bureau of Economic Research.
  21. AgentCoder: Multi-Agent-based Code Generation with Iterative Testing and Optimisation. arXiv preprint arXiv:2312.13010 (2023).
  22. Stephen H Kan. 2003. Metrics and models in software quality engineering. Addison-Wesley Professional.
  23. ChatGPT for good? On opportunities and challenges of large language models for education. Learning and individual differences 103 (2023), 102274.
  24. The langroid contributors. 2024. langroid: A Language Model Framework. https://github.com/langroid/langroid. Accessed: 2024-03-20.
  25. Craig Larman. 2004. Agile and iterative development: a manager’s guide. Addison-Wesley Professional.
  26. Camel: Communicative agents for” mind” exploration of large language model society. Advances in Neural Information Processing Systems 36 (2024).
  27. Encouraging divergent thinking in large language models through multi-agent debate. arXiv preprint arXiv:2305.19118 (2023).
  28. David Lo. 2023. Trustworthy and Synergistic Artificial Intelligence for Software Engineering: Vision and Roadmaps. arXiv preprint arXiv:2309.04142 (2023).
  29. Pattie Maes. 1993. Modeling adaptive autonomous agents. Artificial life 1, 1_2 (1993), 135–162.
  30. Alfred R Mele. 2001. Autonomous agents: From self-control to autonomy. Oxford University Press, USA.
  31. Anton Osika. 2023. gpt-engineer. https://github.com/gpt-engineer-org/gpt-engineer
  32. Communicative agents for software development. arXiv preprint arXiv:2307.07924 (2023).
  33. Yoav Shoham. 1993. Agent-oriented programming. Artificial intelligence 60, 1 (1993), 51–92.
  34. Significant Gravitas. [n. d.]. AutoGPT. https://github.com/Significant-Gravitas/AutoGPT
  35. Jeff Tian. 2005. Software quality engineering: testing, quality assurance, and quantifiable improvement. John Wiley & Sons.
  36. Rainer Unland. 2015. Software agent systems. In Industrial Agents. Elsevier, 3–22.
  37. Voyager: An open-ended embodied agent with large language models. arXiv preprint arXiv:2305.16291 (2023).
  38. A Survey on Large Language Model based Autonomous Agents. CoRR abs/2308.11432 (2023).
  39. Describe, explain, plan and select: Interactive planning with large language models enables open-world multi-task agents. arXiv preprint arXiv:2302.01560 (2023).
  40. Peter Wegner. 1990. Concepts and paradigms of object-oriented programming. ACM Sigplan Oops Messenger 1, 1 (1990), 7–87.
  41. AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation Framework. CoRR abs/2308.08155 (2023). https://doi.org/10.48550/ARXIV.2308.08155 arXiv:2308.08155
  42. An empirical study on challenging math problem solving with gpt-4. arXiv preprint arXiv:2306.01337 (2023).
  43. Shall We Talk: Exploring Spontaneous Collaborations of Competing LLM Agents. arXiv preprint arXiv:2402.12327 (2024).
  44. The rise and potential of large language model based agents: A survey. arXiv preprint arXiv:2309.07864 (2023).
  45. OpenAgents: An Open Platform for Language Agents in the Wild. arXiv:2310.10634 [cs.CL]
  46. APIDocBooster: An Extract-Then-Abstract Framework Leveraging Large Language Models for Augmenting API Documentation. arXiv preprint arXiv:2312.10934 (2023).
  47. Siren’s song in the AI ocean: a survey on hallucination in large language models. arXiv preprint arXiv:2309.01219 (2023).
  48. Hierarchical Auto-Organizing System for Open-Ended Multi-Agent Navigation. arXiv preprint arXiv:2403.08282 (2024).
  49. CCBERT: Self-Supervised Code Change Representation Learning. In 2023 IEEE International Conference on Software Maintenance and Evolution (ICSME). IEEE, 182–193.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets