Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
102 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Automated Questions About Learners' Own Code Help to Detect Fragile Knowledge (2306.16267v1)

Published 28 Jun 2023 in cs.CY

Abstract: Students are able to produce correctly functioning program code even though they have a fragile understanding of how it actually works. Questions derived automatically from individual exercise submissions (QLC) can probe if and how well the students understand the structure and logic of the code they just created. Prior research studied this approach in the context of the first programming course. We replicate the study on a follow-up programming course for engineering students which contains a recap of general concepts in CS1. The task was the classic rainfall problem which was solved by 90% of the students. The QLCs generated from each passing submission were kept intentionally simple, yet 27% of the students failed in at least one of them. Students who struggled with questions about their own program logic had a lower median for overall course points than students who answered correctly.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (23)
  1. “I’m Not a Computer”: How Identity Informs Value and Expectancy During a Programming Activity. In The Interdisciplinarity of the Learning Sciences, 14th International Conference of the Learning Sciences (Nashville, Tennessee) (ICLS 2020, Vol. 2). International Society of the Learning Sciences (ISLS), USA, 705–708. https://repository.isls.org/handle/1/6733
  2. The Robots Are Coming: Exploring the Implications of OpenAI Codex on Introductory Programming. In Proceedings of the 24th Australasian Computing Education Conference (Virtual Event, Australia) (ACE ’22). Association for Computing Machinery, New York, NY, USA, 10–19. https://doi.org/10.1145/3511861.3511863
  3. Cruz Izu and Claudio Mirolo. 2020. Comparing Small Programs for Equivalence: A Code Comprehension Task for Novice Programmers. In Proceedings of the 2020 ACM Conference on Innovation and Technology in Computer Science Education (Trondheim, Norway) (ITiCSE ’20). Association for Computing Machinery, New York, NY, USA, 466–472. https://doi.org/10.1145/3341525.3387425
  4. Fostering Program Comprehension in Novice Programmers - Learning Activities and Learning Trajectories. In Proceedings of the Working Group Reports on Innovation and Technology in Computer Science Education (Aberdeen, Scotland Uk) (ITiCSE-WGR ’19). Association for Computing Machinery, New York, NY, USA, 27–52. https://doi.org/10.1145/3344429.3372501
  5. Cazembe Kennedy and Eileen T. Kraemer. 2019. Qualitative Observations of Student Reasoning: Coding in the Wild. In Proceedings of the 2019 ACM Conference on Innovation and Technology in Computer Science Education (Aberdeen, Scotland Uk) (ITiCSE ’19). Association for Computing Machinery, New York, NY, USA, 224–230. https://doi.org/10.1145/3304221.3319751
  6. Viraj Kumar. 2021. Refute: An Alternative to ‘Explain in Plain English’ Questions. In Proceedings of the 17th ACM Conference on International Computing Education Research (Virtual Event, USA) (ICER 2021). Association for Computing Machinery, New York, NY, USA, 438–440. https://doi.org/10.1145/3446871.3469791
  7. Automated Questionnaires About Students’ JavaScript Programs: Towards Gauging Novice Programming Processes. In Proceedings of the 25th Australasian Computing Education Conference (Melbourne, VIC, Australia) (ACE ’23). Association for Computing Machinery, New York, NY, USA, 49–58. https://doi.org/10.1145/3576123.3576129
  8. Students Struggle to Explain Their Own Program Code. In Proceedings of the 26th ACM Conference on on Innovation and Technology in Computer Science Education V. 1 (Virtual Event, Germany) (ITiCSE ’21). Association for Computing Machinery, New York, NY, USA, 206–212. https://doi.org/10.1145/3430665.3456322
  9. Let’s Ask Students About Their Programs, Automatically. In 2021 IEEE/ACM 29th International Conference on Program Comprehension (ICPC). IEEE, New York, NY, USA, 467–475. https://doi.org/10.1109/ICPC52881.2021.00054
  10. A Multi-National Study of Reading and Tracing Skills in Novice Programmers. In Working Group Reports from ITiCSE on Innovation and Technology in Computer Science Education (Leeds, United Kingdom) (ITiCSE-WGR ’04). Association for Computing Machinery, New York, NY, USA, 119–150. https://doi.org/10.1145/1044550.1041673
  11. Sandra Madison and James Gifford. 2002. Modular Programming. Journal of Research on Technology in Education 34, 3 (2002), 217–229. https://doi.org/10.1080/15391523.2002.10782346
  12. High-School Students’ Mastery of Basic Flow-Control Constructs through the Lens of Reversibility. In Proceedings of the 15th Workshop on Primary and Secondary Computing Education (Virtual Event, Germany) (WiPSCE ’20). Association for Computing Machinery, New York, NY, USA, Article 15, 10 pages. https://doi.org/10.1145/3421590.3421603
  13. D. N. Perkins and Fay Martin. 1986. Fragile Knowledge and Neglected Strategies in Novice Programmers. In Papers Presented at the First Workshop on Empirical Studies of Programmers (Washington, D.C., USA). Ablex Publishing Corp., USA, 213–229.
  14. Arun Raman and Viraj Kumar. 2022. Programming Pedagogy and Assessment in the Era of AI/ML: A Position Paper. In Proceedings of the 15th Annual ACM India Compute Conference (Jaipur, India) (COMPUTE ’22). Association for Computing Machinery, New York, NY, USA, 29–34. https://doi.org/10.1145/3561833.3561843
  15. Kay Sambell and Liz McDowell. 1998. The Construction of the Hidden Curriculum: messages and meanings in the assessment of student learning. Assessment & Evaluation in Higher Education 23, 4 (1998), 391–402. https://doi.org/10.1080/0260293980230406
  16. Jask: Generation of Questions About Learners’ Code in Java. In Proceedings of the 27th ACM Conference on on Innovation and Technology in Computer Science Education Vol. 1 (Dublin, Ireland) (ITiCSE ’22). Association for Computing Machinery, New York, NY, USA, 117–123. https://doi.org/10.1145/3502718.3524761
  17. Carsten Schulte. 2008. Block Model: An Educational Model of Program Comprehension as a Tool for a Scholarly Approach to Teaching. In Proceedings of the Fourth International Workshop on Computing Education Research (Sydney, Australia) (ICER ’08). Association for Computing Machinery, New York, NY, USA, 149–160. https://doi.org/10.1145/1404520.1404535
  18. An Introduction to Program Comprehension for Computer Science Educators. In Proceedings of the 2010 ITiCSE Working Group Reports (Ankara, Turkey) (ITiCSE-WGR ’10). Association for Computing Machinery, New York, NY, USA, 65–86. https://doi.org/10.1145/1971681.1971687
  19. Performing Tasks Can Improve Program Comprehension Mental Model of Novice Developers: An Empirical Approach. In Proceedings of the 28th International Conference on Program Comprehension (Seoul, Republic of Korea) (ICPC ’20). Association for Computing Machinery, New York, NY, USA, 263–273. https://doi.org/10.1145/3387904.3389277
  20. Simon. 2011. Assignment and Sequence: Why Some Students Can’t Recognise a Simple Swap. In Proceedings of the 11th Koli Calling International Conference on Computing Education Research (Koli, Finland) (Koli Calling ’11). Association for Computing Machinery, New York, NY, USA, 10–15. https://doi.org/10.1145/2094131.2094134
  21. What do novices know about programming? Directions in Human-Computer Interaction 6 (1982), 27–54.
  22. Dieter Vogts. 2009. Plagiarising of Source Code by Novice Programmers a ”Cry for Help”?. In Proceedings of the 2009 Annual Research Conference of the South African Institute of Computer Scientists and Information Technologists (Vanderbijlpark, Emfuleni, South Africa) (SAICSIT ’09). Association for Computing Machinery, New York, NY, USA, 141–149. https://doi.org/10.1145/1632149.1632168
  23. ”It Depends on Whether or Not I’m Lucky” How Students in an Introductory Programming Course Discover, Select, and Assess the Utility of Web-Based Resources. In Proceedings of the 27th ACM Conference on on Innovation and Technology in Computer Science Education Vol. 1 (Dublin, Ireland) (ITiCSE ’22). Association for Computing Machinery, New York, NY, USA, 512–518. https://doi.org/10.1145/3502718.3524751
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (3)
  1. Teemu Lehtinen (4 papers)
  2. Otto Seppälä (2 papers)
  3. Ari Korhonen (1 paper)
Citations (4)
View on Semantic Scholar