Mining patterns in syntax trees to automate code reviews of student solutions for programming exercises (2405.01579v1)
Abstract: In programming education, providing manual feedback is essential but labour-intensive, posing challenges in consistency and timeliness. We introduce ECHO, a machine learning method to automate the reuse of feedback in educational code reviews by analysing patterns in abstract syntax trees. This study investigates two primary questions: whether ECHO can predict feedback annotations to specific lines of student code based on previously added annotations by human reviewers (RQ1), and whether its training and prediction speeds are suitable for using ECHO for real-time feedback during live code reviews by human reviewers (RQ2). Our results, based on annotations from both automated linting tools and human reviewers, show that ECHO can accurately and quickly predict appropriate feedback annotations. Its efficiency in processing and its flexibility in adapting to feedback patterns can significantly reduce the time and effort required for manual feedback provisioning in educational settings.
- Ala-Mutka, K. M. (2005). A Survey of Automated Assessment Approaches for Programming Assignments. Computer Science Education, 15(2):83–102.
- Efficient Substructure Discovery from Large Semi-Structured Data. IEICE TRANSACTIONS on Information and Systems, E87-D(12):2754–2763.
- Machine learning based feedback on textual student answers in large courses. Computers and Education: Artificial Intelligence, 3:100081.
- Assessment and Classroom Learning. Assessment in Education: Principles, Policy & Practice, 5(1):7–74.
- Tree-sitter/tree-sitter: V0.20.9. Zenodo.
- Educators’ perceptions of automated feedback systems. Australasian Journal of Educational Technology, 24(4).
- The Power of Feedback. Review of Educational Research, 77(1):81–112.
- Review of recent systems for automatic assessment of programming assignments. In Proceedings of the 10th Koli Calling International Conference on Computing Education Research, Koli Calling ’10, pages 86–93, New York, NY, USA. Association for Computing Machinery.
- Lee, A. V. Y. (2023). Supporting students’ generation of feedback in large-scale online course with artificial intelligence-enabled evaluation. Studies in Educational Evaluation, 77:101250.
- Extracting unit tests from patterns mined in student code to provide improved feedback in autograders. In Seminar Series on Advanced Techniques & Tools for Software Evolution (SATToSE).
- The good, the bad, and the ugly: Mining for patterns in student source code. In Proceedings of the 3rd International Workshop on Education through Advanced Software Engineering and Artificial Intelligence, EASEAI 2021, pages 1–8, New York, NY, USA. Association for Computing Machinery.
- Atomic, reusable feedback: A semi-automated solution for assessing handwritten tasks? A crossover experiment with mathematics teachers. Computers and Education Open, 3:100086.
- Automated Assessment in Computer Science Education: A State-of-the-Art Review. ACM Transactions on Computing Education, 22(3):34:1–34:40.
- Mining Patterns in Source Code Using Tree Mining Algorithms. In Kralj Novak, P., Šmuc, T., and Džeroski, S., editors, Discovery Science, Lecture Notes in Computer Science, pages 471–480, Cham. Springer International Publishing.
- Supporting the Semi-automatic Feedback Provisioning on Programming Assignments. In Temperini, M., Scarano, V., Marenzi, I., Kravcik, M., Popescu, E., Lanzilotti, R., Gennari, R., De La Prieta, F., Di Mascio, T., and Vittorini, P., editors, Methodologies and Intelligent Systems for Technology Enhanced Learning, 12th International Conference, pages 13–19, Cham. Springer International Publishing.
- Tuck, J. (2012). Feedback-giving as social practice: Teachers’ perspectives on feedback as institutional requirement, work and dialogue. Teaching in Higher Education, 17(2):209–221.
- Dodona: Learn to code with a virtual co-teacher that supports active learning. SoftwareX, 24:101578.
- An AI-Based System for Formative and Summative Assessment in Data Science Courses. International Journal of Artificial Intelligence in Education, 31(2):159–185.
- Zaki, M. (2005). Efficiently mining frequent trees in a forest: Algorithms and applications. IEEE Transactions on Knowledge and Data Engineering, 17(8):1021–1035.
- Charlotte Van Petegem (4 papers)
- Kasper Demeyere (1 paper)
- Rien Maertens (3 papers)
- Niko Strijbol (3 papers)
- Bram De Wever (2 papers)
- Bart Mesuere (4 papers)
- Peter Dawyndt (4 papers)