Autonomous Agents in Software Development: A Vision Paper (2311.18440v1)
Abstract: LLMs (LLM) and Generative Pre-trained Transformers (GPT), are reshaping the field of Software Engineering (SE). They enable innovative methods for executing many software engineering tasks, including automated code generation, debugging, maintenance, etc. However, only a limited number of existing works have thoroughly explored the potential of GPT agents in SE. This vision paper inquires about the role of GPT-based agents in SE. Our vision is to leverage the capabilities of multiple GPT agents to contribute to SE tasks and to propose an initial road map for future work. We argue that multiple GPT agents can perform creative and demanding tasks far beyond coding and debugging. GPT agents can also do project planning, requirements engineering, and software design. These can be done through high-level descriptions given by the human developer. We have shown in our initial experimental analysis for simple software (e.g., Snake Game, Tic-Tac-Toe, Notepad) that multiple GPT agents can produce high-quality code and document it carefully. We argue that it shows a promise of unforeseen efficiency and will dramatically reduce lead-times. To this end, we intend to expand our efforts to understand how we can scale these autonomous capabilities further.
- Towards human-bot collaborative software architecting with chatgpt. In Proceedings of the 27th International Conference on Evaluation and Assessment in Software Engineering. 279–285.
- Ömer Aydın and Enis Karaarslan. 2023. Is ChatGPT leading generative AI? What is beyond expectations? What is beyond expectations (2023).
- David Baidoo-Anu and Leticia Owusu Ansah. 2023. Education in the era of generative artificial intelligence (AI): Understanding the potential benefits of ChatGPT in promoting teaching and learning. Available at SSRN 4337484 (2023).
- Grounded copilot: How programmers interact with code-generating models. Proceedings of the ACM on Programming Languages 7, OOPSLA1 (2023), 85–111.
- Language models are few-shot learners. Advances in neural information processing systems 33 (2020), 1877–1901.
- Paweł Budzianowski and Ivan Vulić. 2019. Hello, it’s GPT-2–how can I help you? towards the use of pretrained language models for task-oriented dialogue systems. arXiv preprint arXiv:1907.05774 (2019).
- Alessio Buscemi. 2023. A Comparative Study of Code Generation using ChatGPT 3.5 across 10 Programming Languages. arXiv preprint arXiv:2308.04477 (2023).
- A comprehensive survey of ai-generated content (aigc): A history of generative ai from gan to chatgpt. arXiv preprint arXiv:2303.04226 (2023).
- KR1442 Chowdhary and KR Chowdhary. 2020. Natural language processing. Fundamentals of artificial intelligence (2020), 603–649.
- Robert Dale. 2021. GPT-3: What’s it good for? Natural Language Engineering 27, 1 (2021), 113–118.
- Self-collaboration Code Generation via ChatGPT. arXiv preprint arXiv:2304.07590 (2023).
- CODEP: grammatical seq2seq model for general-purpose code generation. In Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis. 188–198.
- Investigating Code Generation Performance of Chat-GPT with Crowdsourcing Social Data. In Proceedings of the 47th IEEE Computer Software and Applications Conference. 1–10.
- Luciano Floridi and Massimo Chiriatti. 2020. GPT-3: Its nature, scope, limits, and consequences. Minds and Machines 30 (2020), 681–694.
- Regulating ChatGPT and other large generative AI models. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency. 1112–1123.
- Adam Hörnemalm. 2023. ChatGPT as a Software Development Tool: The Future of Development.
- Cure: Code-aware neural machine translation for automatic program repair. In 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE). IEEE, 1161–1173.
- Junaed Younus Khan and Gias Uddin. 2022. Automatic code documentation generation using gpt-3. In Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering. 1–6.
- Is your code generated by chatgpt really correct? rigorous evaluation of large language models for code generation. arXiv preprint arXiv:2305.01210 (2023).
- GPT understands, too. AI Open (2023).
- The Scope of ChatGPT in Software Engineering: A Thorough Investigation. arXiv preprint arXiv:2305.12138 (2023).
- Comparing Software Developers with ChatGPT: An Empirical Investigation. arXiv preprint arXiv:2305.11837 (2023).
- Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems 35 (2022), 27730–27744.
- Instruction tuning with gpt-4. arXiv preprint arXiv:2304.03277 (2023).
- Improving language understanding by generative pre-training. (2018).
- Language models are unsupervised multitask learners. OpenAI blog 1, 8 (2019), 9.
- Incorporating domain knowledge through task augmentation for front-end JavaScript code generation. In Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 1533–1543.
- Understanding emails and drafting responses–An approach using GPT-3. arXiv preprint arXiv:2102.03062 (2021).
- Is ChatGPT the Ultimate Programming Assistant–How far is it? arXiv preprint arXiv:2304.11938 (2023).
- Exploring transformers in natural language generation: Gpt, bert, and xlnet. arXiv preprint arXiv:2102.08036 (2021).
- Christoph Treude. 2023. Navigating Complexity in Software Engineering: A Prototype for Comparing GPT-n Solutions. arXiv preprint arXiv:2301.12169 (2023).
- Expectation vs. experience: Evaluating the usability of code generation tools powered by large language models. In Chi conference on human factors in computing systems extended abstracts. 1–7.
- Attention is all you need. Advances in neural information processing systems 30 (2017).
- Chatgpt prompt patterns for improving code quality, refactoring, requirements elicitation, and software design. arXiv preprint arXiv:2303.07839 (2023).
- Mingyu Zong and Bhaskar Krishnamachari. 2022. A survey on GPT-3. arXiv preprint arXiv:2212.00857 (2022).
- Zeeshan Rasheed (23 papers)
- Muhammad Waseem (66 papers)
- Kai-Kristian Kemell (36 papers)
- Wang Xiaofeng (2 papers)
- Anh Nguyen Duc (16 papers)
- Kari Systä (11 papers)
- Pekka Abrahamsson (105 papers)