Toward Educator-focused Automated Scoring Systems for Reading and Writing (2112.11973v1)

Abstract: This paper presents methods for improving automated essay scoring with techniques that address the computational trade-offs of self-attention and document length. To make Automated Essay Scoring (AES) more useful to practitioners, researchers must overcome the challenges of data and label availability, authentic and extended writing, domain scoring, prompt and source variety, and transfer learning. This paper addresses these challenges using neural network models by employing techniques that preserve essay length as an important feature without increasing model training costs. It introduces techniques for minimizing classification loss on ordinal labels using multi-objective learning, capturing semantic information across the entire essay using sentence embeddings to use transformer architecture across arbitrarily long documents, the use of such models for transfer learning, automated hyperparameter generation based on prompt-corpus metadata, and, most importantly, the use of semantic information to provide meaningful insights into student reading through analysis of passage-dependent writing resulting in state-of-the-art results for various essay tasks.