Automated Essay Scoring in Argumentative Writing: DeBERTeachingAssistant (2307.04276v1)
Abstract: Automated Essay scoring has been explored as a research and industry problem for over 50 years. It has drawn a lot of attention from the NLP community because of its clear educational value as a research area that can engender the creation of valuable time-saving tools for educators around the world. Yet, these tools are generally focused on detecting good grammar, spelling mistakes, and organization quality but tend to fail at incorporating persuasiveness features in their final assessment. The responsibility to give actionable feedback to the student to improve the strength of their arguments is left solely on the teacher's shoulders. In this work, we present a transformer-based architecture capable of achieving above-human accuracy in annotating argumentative writing discourse elements for their persuasiveness quality and we expand on planned future work investigating the explainability of our model so that actionable feedback can be offered to the student and thus potentially enable a partnership between the teacher's advice and the machine's advice.
- Y. Attali, J. Burstein, Automated essay scoring with e-rater® v. 2.0, ETS Research Report Series 2004 (2004) i–21.
- E. B. Page, The imminence of… grading essays by computer, The Phi Delta Kappan 47 (1966) 238–243.
- The persuasive essays for rating, selecting, and understanding argumentative and discourse elements (persuade) corpus 1.0, Assessing Writing 54 (2022) 100667.
- C. Stab, I. Gurevych, Identifying argumentative discourse structures in persuasive essays, in: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), 2014, pp. 46–56.
- I. Persing, V. Ng, Modeling argument strength in student essays, in: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), 2015, pp. 543–552.
- Give me more feedback: Annotating argument persuasiveness and related attributes in student essays, in: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2018, pp. 621–631.
- Learning to give feedback: Modeling attributes affecting argument persuasiveness in student essays., in: IJCAI, 2018, pp. 4130–4136.
- Automatic argument quality assessment–new datasets and methods, arXiv preprint arXiv:1909.01007 (2019).
- Deberta: Decoding-enhanced bert with disentangled attention, arXiv preprint arXiv:2006.03654 (2020).
- Mixed precision training, arXiv preprint arXiv:1710.03740 (2017).
- Training deep nets with sublinear memory cost, arXiv preprint arXiv:1604.06174 (2016).
- L. Breiman, Bagging predictors, Machine learning 24 (1996) 123–140.
- Lightgbm: A highly efficient gradient boosting decision tree, Advances in neural information processing systems 30 (2017).
- Understanding bag-of-words model: a statistical framework, International journal of machine learning and cybernetics 1 (2010) 43–52.
- R. E. Schapire, A brief introduction to boosting, in: Ijcai, volume 99, 1999, pp. 1401–1406.
- B. Pavlyshenko, Using stacking approaches for machine learning models, in: 2018 IEEE Second International Conference on Data Stream Mining & Processing (DSMP), IEEE, 2018, pp. 255–258.
- A novel stacking approach for accurate detection of fake news, IEEE Access 9 (2021) 22626–22639.
- Adversarial weight perturbation helps robust generalization, Advances in Neural Information Processing Systems 33 (2020) 2958–2969.
- Robust weight perturbation for adversarial training, arXiv preprint arXiv:2205.14826 (2022).
- Virtual adversarial training: a regularization method for supervised and semi-supervised learning, IEEE transactions on pattern analysis and machine intelligence 41 (2018) 1979–1993.
- The ai index 2021 annual report, arXiv preprint arXiv:2103.06312 (2021).
- Superglue: A stickier benchmark for general-purpose language understanding systems, Advances in neural information processing systems 32 (2019).