Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
41 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
41 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

AI-Tutoring in Software Engineering Education (2404.02548v2)

Published 3 Apr 2024 in cs.SE and cs.AI

Abstract: With the rapid advancement of AI in various domains, the education sector is set for transformation. The potential of AI-driven tools in enhancing the learning experience, especially in programming, is immense. However, the scientific evaluation of LLMs used in Automated Programming Assessment Systems (APASs) as an AI-Tutor remains largely unexplored. Therefore, there is a need to understand how students interact with such AI-Tutors and to analyze their experiences. In this paper, we conducted an exploratory case study by integrating the GPT-3.5-Turbo model as an AI-Tutor within the APAS Artemis. Through a combination of empirical data collection and an exploratory survey, we identified different user types based on their interaction patterns with the AI-Tutor. Additionally, the findings highlight advantages, such as timely feedback and scalability. However, challenges like generic responses and students' concerns about a learning progress inhibition when using the AI-Tutor were also evident. This research adds to the discourse on AI's role in education.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (34)
  1. J. R Anderson and E. Skwarecki. 1986. The automated tutoring of introductory computer programming. Commun. ACM 29, 9 (sep 1986), 842–849. https://doi.org/10.1145/6592.6593
  2. Luka Bradeško and Dunja Mladenić. 2012. A survey of chatbot systems through a loebner prize competition. In Proceedings of Slovenian language technologies society eighth conference of language technologies, Vol. 2. sn, 34–37.
  3. PL Brusilovsky. 1992. Intelligent tutor, environment and manual for introductory programming. Educational & Training Technology International 29, 1 (1992), 26–34.
  4. Intelligent tutoring systems for programming education. In Proceedings of the 20th Australasian Computing Education Conference. ACM. https://doi.org/10.1145/3160489.3160492
  5. Marian Daun and Jennifer Brings. 2023. How ChatGPT Will Change Software Engineering Education. In Proceedings of the Conference on Innovation and Technology in Computer Science Education V. 1. 110–116.
  6. Fred D. Davis. 1985. A Technology Acceptance Model for Empirically Testing New End-User Information Systems: Theory and Results. Ph. D. Dissertation. Massachusetts Institute of Technology, Sloan School of Management.
  7. Accelerating business growth with big data and artificial intelligence. In 2020 Fourth International Conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud)(I-SMAC). IEEE, 441–448.
  8. Andreas M Hinz. 1992. Pascal’s Triangle and the Tower of Hanoi. The American mathematical monthly 99, 6 (1992), 538–544.
  9. Enabling technologies for personalized and precision medicine. Trends in biotechnology 38, 5 (2020), 497–518.
  10. J-LATTE: a Constraint-based Tutor for Java. (2009).
  11. Chatgpt and software testing education: Promises & perils. In 2023 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW). IEEE, 4130–4137.
  12. ChatGPT for good? On opportunities and challenges of large language models for education. Learning and Individual Differences 103 (apr 2023), 102274. https://doi.org/10.1016/j.lindif.2023.102274
  13. Towards a systematic review of automated feedback generation for programming exercises. In Proceedings of the 2016 ACM Conference on Innovation and Technology in Computer Science Education. 41–46.
  14. Stephan Krusche. 2021. Interactive learning - A scalable and adaptive learning approach for large courses. Habilitation. Technische Universität München.
  15. Stephan Krusche and Andreas Seitz. 2018. Artemis: An automatic assessment management system for interactive learning. In Proceedings of the 49th ACM technical symposium on computer science education. 284–289.
  16. Nir Kshetri. 2023. The Economics of Generative Artificial Intelligence in the Academic Industry. Computer 56, 8 (aug 2023), 77–83. https://doi.org/10.1109/mc.2023.3278089
  17. Interacting with educational chatbots: A systematic review. Education and Information Technologies 28, 1 (2023), 973–1018.
  18. James A Kulik and JD Fletcher. 2016. Effectiveness of intelligent tutoring systems: a meta-analytic review. Review of educational research 86, 1 (2016), 42–78.
  19. Constructive Alignment in Modern Computing Education: An Open-Source Computer-Based Examination System. In 23nd Koli Calling International Conference on Computing Education Research (Koli, Finland) (Koli ’23). https://doi.org/10.35542/osf.io/nmpf6
  20. Jailbreaking chatgpt via prompt engineering: An empirical study. arXiv preprint arXiv:2305.13860 (2023).
  21. Chung Kwan Lo. 2023. What is the impact of ChatGPT on education? A rapid review of the literature. Education Sciences 13, 4 (2023), 410.
  22. ChatGPT, Can You Generate Solutions for my Coding Exercises? An Evaluation on its Effectiveness in an undergraduate Java Programming Course. arXiv preprint arXiv:2305.13680 (2023).
  23. Zachary A Pardos and Shreya Bhandari. 2023. Learning gain differences between ChatGPT and human tutor generated algebra hints. arXiv preprint arXiv:2302.06871 (2023).
  24. Mark Petticrew and Helen Roberts. 2008. Systematic reviews in the social sciences: A practical guide. John Wiley & Sons.
  25. Partha Pratim Ray. 2023. ChatGPT: A comprehensive review on background, applications, key challenges, bias, ethics, limitations and future scope. Internet of Things and Cyber-Physical Systems (2023).
  26. ChatGPT: Bullshit spewer or the end of traditional assessments in higher education? Journal of Applied Learning and Teaching 6, 1 (2023).
  27. Lecturers’ and Students’ Experiences with an Automated Programming Assessment System. (2023).
  28. An analysis of the automatic bug fixing performance of chatgpt. arXiv preprint arXiv:2301.08653 (2023).
  29. Ch (e) atgpt? an Anecdotal Approach on the Impact of Chatgpt on Teaching and Learning Giscience. Preprint) https://doi. org/10.35542/osf. io/j3m9b (2023).
  30. Nigar M Shafiq Surameery and Mohammed Y Shakor. 2023. Use chat gpt to solve programming bugs. International Journal of Information Technology & Computer Engineering (IJITC) ISSN: 2455-5290 3, 01 (2023), 17–22.
  31. Is ChatGPT the Ultimate Programming Assistant–How far is it? arXiv preprint arXiv:2304.11938 (2023).
  32. Rainer Winkler and Matthias Söllner. 2018. Unleashing the potential of chatbots in education: A state-of-the-art analysis. In Academy of Management Proceedings, Vol. 2018. Academy of Management Briarcliff Manor, NY 10510, 15903.
  33. Experimentation in software engineering. Springer Science & Business Media.
  34. Caiming Zhang and Yang Lu. 2021. Study on artificial intelligence: The state of the art and future prospects. Journal of Industrial Information Integration 23 (2021), 100224.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (5)
  1. Eduard Frankford (4 papers)
  2. Clemens Sauerwein (10 papers)
  3. Patrick Bassner (3 papers)
  4. Stephan Krusche (17 papers)
  5. Ruth Breu (15 papers)
Citations (6)
View on Semantic Scholar