Anomaly Detection in Scratch Assignments

Abstract: For teachers, automated tool support for debugging and assessing their students' programming assignments is a great help in their everyday business. For block-based programming languages which are commonly used to introduce younger learners to programming, testing frameworks and other software analysis tools exist, but require manual work such as writing test suites or formal specifications. However, most of the teachers using languages like Scratch are not trained for or experienced in this kind of task. Linters do not require manual work but are limited to generic bugs and therefore miss potential task-specific bugs in student solutions. In prior work, we proposed the use of anomaly detection to find project-specific bugs in sets of student programming assignments automatically, without any additional manual labour required from the teachers' side. Evaluation on student solutions for typical programming assignments showed that anomaly detection is a reliable way to locate bugs in a data set of student programs. In this paper, we enhance our initial approach by lowering the abstraction level. The results suggest that the lower abstraction level can focus anomaly detection on the relevant parts of the programs.