2000 character limit reached
Automatic Scoring of Students' Science Writing Using Hybrid Neural Network
Published 2 Dec 2023 in cs.CL | (2312.03752v2)
Abstract: This study explores the efficacy of a multi-perspective hybrid neural network (HNN) for scoring student responses in science education with an analytic rubric. We compared the accuracy of the HNN model with four ML approaches (BERT, AACR, Naive Bayes, and Logistic Regression). The results have shown that HHN achieved 8%, 3%, 1%, and 0.12% higher accuracy than Naive Bayes, Logistic Regression, AACR, and BERT, respectively, for five scoring aspects (p<0.001). The overall HNN's perceived accuracy (M = 96.23%, SD = 1.45%) is comparable to the (training and inference) expensive BERT model's accuracy (M = 96.12%, SD = 1.52%). We also have observed that HNN is x2 more efficient in training and inferencing than BERT and has comparable efficiency to the lightweight but less accurate Naive Bayes model. Our study confirmed the accuracy and efficiency of using HNN to score students' science writing automatically.
- A training algorithm for optimal margin classifiers. In Proceedings of the fifth annual workshop on Computational learning theory, New York, NY, USA, July 1992. ACM.
- National Research Council et al. A framework for K-12 science education: Practices, crosscutting concepts, and core ideas. National Academies Press, 2012.
- National Research Council et al. Next generation science standards: For states, by states. 2013.
- National Research Council et al. Developing assessments for the next generation science standards. 2014.
- Li Deng. Deep learning: Methods and applications. now Publishers Inc, 2014.
- BERT: Pre-training of deep bidirectional transformers for language understanding. October 2018.
- Support vector regression machines. Advances in neural information processing systems, 9, 1996.
- Creating and using instructionally supportive assessments in NGSS classrooms. NSTA Press, 2024.
- Exploring the effect of assessment construct complexity on machine learning scoring of argumentation. In Annual Conference of National Association of Research in Science Teaching, Florida, 2021.
- Guiding teacher use of artificial intelligence-based knowledge-in-use assessment to improve instructional decisions: A conceptual framework. In Xiaoming Zhai and Joseph Krajcik, editors, Uses of Artificial Intelligence in STEM Education, pages xx–xx. Oxford University Press, 2024.
- Comparison of machine learning performance using analytic and holistic coding approaches across constructed response assessments aligned to a science learning progression. J. Sci. Educ. Technol., 30(2):150–167, April 2021a.
- Comparison of machine learning performance using analytic and holistic coding approaches across constructed response assessments aligned to a science learning progression. Journal of Science Education and Technology, 30(2):150–167, 2021b.
- Fine-tuning chatgpt for automatic scoring. arXiv preprint arXiv:2310.10072, 2023.
- Artificial general intelligence (agi) for education. arXiv preprint arXiv:2304.12479, 2023.
- Enhanced hybrid neural network for automated essay scoring. Expert Syst., 39(10), December 2022.
- Validation of automated scoring of science assessments. J. Res. Sci. Teach., 53(2):215–233, February 2016.
- Context matters: A strategy to pre-train language model for science education. January 2023.
- Using machine learning to score multi-dimensional assessments of chemistry and physics. J. Sci. Educ. Technol., 30(2):239–254, April 2021.
- Deep learning–based text classification: a comprehensive review. ACM computing surveys (CSUR), 54(3):1–40, 2021.
- Automated analysis of short responses in an interactive synthetic tutoring system for introductory physics. Phys. Rev. Phys. Educ. Res., 12(1), March 2016.
- Human vs. computer diagnosis of students’ natural selection knowledge: Testing the efficacy of text analytic software. J. Sci. Educ. Technol., 21(1):56–73, February 2012.
- Transforming biology assessment with machine learning: Automated scoring of written evolutionary explanations. J. Sci. Educ. Technol., 21(1):183–196, February 2012.
- John Platt. Sequential minimal optimization: A fast algorithm for training support vector machines. April 1998.
- An empirical investigation of neural methods for content scoring of scientific explanations. In Proceedings of the Fifteenth Workshop on Innovative Use of NLP for Building Educational Applications, pages 135–144. 2020.
- Dual memory network model for sentiment analysis of review text. Knowl. Based Syst., 188(105004):105004, January 2020.
- A hybrid neural networks-fuzzy logic-genetic algorithm for grade estimation. Computers & geosciences, 42:18–27, 2012.
- Student-performulator: Student academic performance using hybrid deep neural network. Sustainability, 13(17):9775, August 2021.
- Xiaoming Zhai. Practices and theories: How can machine learning assist in innovative assessment practices in science education. Journal of Science Education and Technology, 30(2):139–149, 2021.
- Ai and formative assessment: The train has left the station. Journal of Research in Science Teaching, 60(6):1390–1398, 2023. DOI: 10.1002/tea.21885.
- Applying machine learning in science assessment: a systematic review. Studies in Science Education, 56(1):111–151, 2020.
- A meta-analysis of machine learning-based science assessments: Factors impacting machine-human score agreements. J. Sci. Educ. Technol., 30(3):361–379, June 2021.
- Applying machine learning to automatically assess scientific models. J. Res. Sci. Teach., 59(10):1765–1794, December 2022.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.