2000 character limit reached
How LLMs Aid in UML Modeling: An Exploratory Study with Novice Analysts (2404.17739v2)
Published 27 Apr 2024 in cs.SE
Abstract: Since the emergence of GPT-3, LLMs have caught the eyes of researchers, practitioners, and educators in the field of software engineering. However, there has been relatively little investigation regarding the performance of LLMs in assisting with requirements analysis and UML modeling. This paper explores how LLMs can assist novice analysts in creating three types of typical UML models: use case models, class diagrams, and sequence diagrams. For this purpose, we designed the modeling tasks of these three UML models for 45 undergraduate students who participated in a requirements modeling course, with the help of LLMs. By analyzing their project reports, we found that LLMs can assist undergraduate students as novice analysts in UML modeling tasks, but LLMs also have shortcomings and limitations that should be considered when using them.
- J. Liu, C. S. Xia, Y. Wang, and L. Zhang, “Is your code generated by chatgpt really correct? rigorous evaluation of large language models for code generation,” Advances in Neural Information Processing Systems, vol. 36, 2024.
- P. Vaithilingam, T. Zhang, and E. L. Glassman, “Expectation vs. experience: Evaluating the usability of code generation tools powered by large language models,” in Chi conference on human factors in computing systems extended abstracts, pp. 1–7, 2022.
- A. Ahmad, M. Waseem, P. Liang, M. Fahmideh, M. S. Aktar, and T. Mikkonen, “Towards human-bot collaborative software architecting with chatgpt,” in Proceedings of the 27th International Conference on Evaluation and Assessment in Software Engineering (EASE), pp. 279–285, 2023.
- D. Zimmermann and A. Koziolek, “Automating gui-based software testing with gpt-3,” in 2023 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW), pp. 62–65, IEEE, 2023.
- A. v. Lamsweerde, Requirements Engineering: From System Goals to UML Models to Software Specifications. John Wiley & Sons, Ltd, 2009.
- B. Wang, C. Wang, P. Liang, B. Li, and C. Zeng, “Case Study for the Paper: How LLMs Aid in UML Modeling: An Exploratory Study with Novice Analysts,” January 2024. https://zenodo.org/doi/10.5281/zenodo.10532600.
- Z. Zheng, K. Ning, J. Chen, Y. Wang, W. Chen, L. Guo, and W. Wang, “Towards an understanding of large language models in software engineering tasks,” arXiv preprint arXiv:2308.11396, 2023.
- D. Luitel, S. Hassani, and M. Sabetzadeh, “Improving requirements completeness: Automated assistance through large language models,” arXiv preprint arXiv:2308.03784, 2023.
- J. White, S. Hays, Q. Fu, J. Spencer-Smith, and D. C. Schmidt, “Chatgpt prompt patterns for improving code quality, refactoring, requirements elicitation, and software design,” arXiv preprint arXiv:2303.07839, 2023.
- J. Zhang, Y. Chen, N. Niu, and C. Liu, “Evaluation of chatgpt on requirements information retrieval under zero-shot setting,” Available at SSRN 4450322, 2023.
- D. Xie, B. Yoo, N. Jiang, M. Kim, L. Tan, X. Zhang, and J. S. Lee, “Impact of large language models on generating software specifications,” arXiv preprint arXiv:2306.03324, 2023.
- J. Jeuring, R. Groot, and H. Keuning, “What skills do you need when developing software using chatgpt? (discussion paper),” arXiv preprint arXiv:2310.05998, 2023.
- M. Waseem, T. Das, A. Ahmad, M. Fehmideh, P. Liang, and T. Mikkonen, “Chatgpt as a software development bot: A project-based study,” in Proceedings of the 19th International Conference on Evaluation of Novel Approaches to Software Engineering (ENASE), 2024.
- C. Arora, J. Grundy, and M. Abdelrazek, “Advancing requirements engineering through generative ai: Assessing the role of llms,” arXiv preprint arXiv:2310.13976, 2023.
- A. R. Sadik, S. Brulin, and M. Olhofer, “Coding by design: Gpt-4 empowers agile model driven development,” arXiv preprint arXiv:2310.04304, 2023.
- H. Kanuka, G. Koreki, R. Soga, and K. Nishikawa, “Exploring the chatgpt approach for bidirectional traceability problem between design models and code,” arXiv preprint arXiv:2309.14992, 2023.
- J. Cámara, J. Troya, L. Burgueño, and A. Vallecillo, “On the assessment of generative ai in modeling tasks: an experience report with chatgpt and uml,” Software and Systems Modeling, pp. 1–13, 2023.
- C. Larman, Applying UML and Patterns: An Introduction to Object Oriented Analysis and Design and Interative Development. Pearson Education, 2012.
- Beian Wang (2 papers)
- Chong Wang (308 papers)
- Peng Liang (94 papers)
- Bing Li (374 papers)
- Cheng Zeng (32 papers)