Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
38 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
41 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

ChatGPT in the Classroom: An Analysis of Its Strengths and Weaknesses for Solving Undergraduate Computer Science Questions (2304.14993v4)

Published 28 Apr 2023 in cs.HC, cs.AI, and cs.CY

Abstract: ChatGPT is an AI LLM developed by OpenAI that can understand and generate human-like text. It can be used for a variety of use cases such as language generation, question answering, text summarization, chatbot development, language translation, sentiment analysis, content creation, personalization, text completion, and storytelling. While ChatGPT has garnered significant positive attention, it has also generated a sense of apprehension and uncertainty in academic circles. There is concern that students may leverage ChatGPT to complete take-home assignments and exams and obtain favorable grades without genuinely acquiring knowledge. This paper adopts a quantitative approach to demonstrate ChatGPT's high degree of unreliability in answering a diverse range of questions pertaining to topics in undergraduate computer science. Our analysis shows that students may risk self-sabotage by blindly depending on ChatGPT to complete assignments and exams. We build upon this analysis to provide constructive recommendations to both students and instructors.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (30)
  1. Reactions: Princeton faculty discuss ChatGPT in the classroom, Feb. 2023.
  2. Will ChatGPT take my job? Here are 20 professions that could be replaced by AI. The Economic Times (Mar. 2023).
  3. Investigating the potential of gpt-3 in providing feedback for programming assessments. In Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education V. 1 (New York, NY, USA, 2023), ITiCSE 2023, Association for Computing Machinery, p. 292–298.
  4. Programming is hard - or at least it used to be: Educational opportunities and challenges of ai code generation. In Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1 (New York, NY, USA, 2023), SIGCSE 2023, Association for Computing Machinery, p. 500–506.
  5. Gpt-3 vs object oriented programming assignments: An experience report. In Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education V. 1 (New York, NY, USA, 2023), ITiCSE 2023, Association for Computing Machinery, p. 61–67.
  6. How chatgpt will change software engineering education. In Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education V. 1 (New York, NY, USA, 2023), ITiCSE 2023, Association for Computing Machinery, p. 110–116.
  7. Conversing with copilot: Exploring prompt engineering for solving cs1 problems using natural language. In Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1 (New York, NY, USA, 2023), SIGCSE 2023, Association for Computing Machinery, p. 1136–1142.
  8. A SWOT analysis of ChatGPT: Implications for educational practice and research. Innovations in Education and Teaching International 0, 0 (Mar. 2023), 1–15. Publisher: Routledge _eprint: https://doi.org/10.1080/14703297.2023.2195846.
  9. The robots are coming: Exploring the implications of openai codex on introductory programming. In Proceedings of the 24th Australasian Computing Education Conference (New York, NY, USA, 2022), ACE ’22, Association for Computing Machinery, p. 10–19.
  10. My ai wants to know if this will be on the exam: Testing openai’s codex on cs2 programming exercises. In Proceedings of the 25th Australasian Computing Education Conference (New York, NY, USA, 2023), ACE ’23, Association for Computing Machinery, p. 97–104.
  11. Chatgpt for Designing Course Outlines: A Boon or Bane to Modern Technology, Mar. 2023.
  12. Who answers it better? an in-depth analysis of chatgpt and stack overflow answers to software engineering questions, 2023.
  13. Does Creating Programming Assignments with Tests Lead to Improved Performance in Writing Unit Tests? In Proceedings of the ACM Conference on Global Computing Education (New York, NY, USA, May 2019), CompEd ’19, Association for Computing Machinery, pp. 106–112.
  14. Comparing code explanations created by students and large language models. In Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education V. 1 (New York, NY, USA, 2023), ITiCSE 2023, Association for Computing Machinery, p. 124–130.
  15. Using large language models to enhance programming error messages. In Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1 (New York, NY, USA, 2023), SIGCSE 2023, Association for Computing Machinery, p. 563–569.
  16. Experiences from using code explanations generated by large language models in a web software development e-book. In Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1 (New York, NY, USA, 2023), SIGCSE 2023, Association for Computing Machinery, p. 931–937.
  17. On the educational impact of chatgpt: Is artificial intelligence ready to obtain a university degree? In Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education V. 1 (New York, NY, USA, 2023), ITiCSE 2023, Association for Computing Machinery, p. 47–53.
  18. Oh, L.-B. Goal Setting and Self-regulated Experiential Learning in a Paired Internship Program. In Proceedings of the ACM Conference on Global Computing Education (New York, NY, USA, May 2019), CompEd ’19, Association for Computing Machinery, p. 239.
  19. OpenAI. Introducing ChatGPT, Nov. 2022.
  20. Chatgpt, can you generate solutions for my coding exercises? an evaluation on its effectiveness in an undergraduate java programming course. In Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education V. 1 (New York, NY, USA, 2023), ITiCSE 2023, Association for Computing Machinery, p. 54–60.
  21. Evaluating the performance of code generation models for solving parsons problems with small prompt variations. In Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education V. 1 (New York, NY, USA, 2023), ITiCSE 2023, Association for Computing Machinery, p. 299–305.
  22. Automatic generation of programming exercises and code explanations using large language models. In Proceedings of the 2022 ACM Conference on International Computing Education Research - Volume 1 (New York, NY, USA, 2022), ICER ’22, Association for Computing Machinery, p. 27–43.
  23. Can generative pre-trained transformers (gpt) pass assessments in higher education programming courses? In Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education V. 1 (New York, NY, USA, 2023), ITiCSE 2023, Association for Computing Machinery, p. 117–123.
  24. Shankland, S. Why We’re Obsessed With the Mind-Blowing ChatGPT AI Chatbot, Feb. 2023.
  25. Impact of Open-Ended Assignments on Student Self-Efficacy in CS1. In Proceedings of the ACM Conference on Global Computing Education (New York, NY, USA, May 2019), CompEd ’19, Association for Computing Machinery, pp. 215–221.
  26. Comparison of Learning Programming Between Interactive Computer Tutors and Human Teachers. In Proceedings of the ACM Conference on Global Computing Education (New York, NY, USA, May 2019), CompEd ’19, Association for Computing Machinery, pp. 2–8.
  27. Taecharungroj, V. “What Can ChatGPT Do?” Analyzing Early Reactions to the Innovative AI Chatbot on Twitter. Big Data and Cognitive Computing 7 (Feb. 2023), 35.
  28. The Learning Network, N. Y. T. What Students Are Saying About ChatGPT - The New York Times, Feb. 2023.
  29. Wermelinger, M. Using github copilot to solve simple programming problems. In Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1 (New York, NY, USA, 2023), SIGCSE 2023, Association for Computing Machinery, p. 172–178.
  30. Developing and Assessing Educational Games to Enhance Cyber Security Learning in Computer Science. In Proceedings of the ACM Conference on Global Computing Education (New York, NY, USA, May 2019), CompEd ’19, Association for Computing Machinery, p. 241.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (8)
  1. Ishika Joshi (5 papers)
  2. Ritvik Budhiraja (5 papers)
  3. Harshal Dev (2 papers)
  4. Jahnvi Kadia (1 paper)
  5. M. Osama Ataullah (1 paper)
  6. Sayan Mitra (50 papers)
  7. Dhruv Kumar (41 papers)
  8. Harshal D. Akolekar (6 papers)
Citations (26)
View on Semantic Scholar