2000 character limit reached
An NLP Crosswalk Between the Common Core State Standards and NAEP Item Specifications (2405.17284v2)
Published 27 May 2024 in cs.CL and cs.AI
Abstract: Natural language processing (NLP) is rapidly developing for applications in educational assessment. In this paper, I describe an NLP-based procedure that can be used to support subject matter experts in establishing a crosswalk between item specifications and content standards. This paper extends recent work by proposing and demonstrating the use of multivariate similarity based on embedding vectors for sentences or texts. In particular, a hybrid regression procedure is demonstrated for establishing the match of each content standard to multiple item specifications. The procedure is used to evaluate the match of the Common Core State Standards (CCSS) for mathematics at grade 4 to the corresponding item specifications for the 2026 National Assessment of Educational Progress (NAEP).
- National Assessment Governing Board. Mathematics Framework for the 2026 National Assessment of Educational Progress. Author, Washington, D.C., 2021.
- NGA Center & CCSSO [National Governors Association Center for Best Practices and Council of Chief State School Officers]. Common Core State Standards for Mathematics. Council of Chief State School Officers, Washington, D.C., 2010.
- E. Forte. Evaluating alignment in large-scale standards-based assessment systems. Council of Chief State School Officers, Washington, D.C., 2017.
- M. S. Smith and J. O’Day. Systemic school reform. Journal of Education Policy, 5(5):233–267, 1990.
- N. L. Webb. Criteria for alignment of expectations and assessments in mathematics and science education (Council of Chief State School Officers and National Institute for Science Education Research Monograph No. 6). Wisconsin Center for Education Research, Madison, WI, 1997.
- A.C. Blank, R.K.and Porter and J. L. Smithson. New tools for analyzing teaching, curriculum and standards in mathematics and science. Council of Chief State School Officers, Washington, DC, 2001.
- Inc. Achieve. An alignment analysis of Washington State’s college readiness mathematic standards with various local placement tests. Author, Cambridge, MA, 2006.
- M. J. Leighton, J. P. & Gierl. Defining and evaluating models of cognition used in educational measurement to make inferences about examinees’ thinking processes. Educational Measurement: Issues and Practice, 26:3–16, 2007.
- A Comparison of the2011 Grade 8 NAEP and TIMSS Mathematics and Science Frameworks (NCES 2013-462). National Center for Education Statistics, Washington, D.C., 2013.
- Study of the alignment of the 2015 NAEP mathematics items at grades 4 and 8 to the Common Core State Standards for Mathematics. National Assessment Governing Board, Washington, D.C., October 2015.
- R. Butterfuss and D. Doran. An application of text embeddings to support alignment of educational content standards. Paper Presented at Generative Artificial Intelligence for Measurement and Education Meeting, feb 2024.
- Catalog: An educational content tagging system. Proceedings of the 14th international conference on educational data mining (EDM 2021), 2021.
- NGSS Lead States. Next Generation Science Standards: For states, by states. National Academies Press, Washington, D.C., 2013.
- Z. Zhou and K.S. Ostrow. Transformer-based automated content-standards alignment: A pilot study. In G. et al. Meiselwitz, editor, HCI International 2022 - Late Breaking Papers. Interaction in New Media, Learning and Games. HCII 2022, Lecture Notes in Computer Science. Springer, Cham., 2022.
- National Assessment Governing Board. Science framework for the 2019 National Assessment of Educational Progress. Author, Washington, D.C., 2013.
- G. Camilli. The 2013-15 decline in naep mathematics in grade 4: What it teaches us about naep. measurement: Interdisciplinary research and perspectives. Measurement: Interdisciplinary Research and Perspectives, 19(4):236–245, 2021.
- National Assessment Governing Board. Draft Assessment and Item Specifications for the 2026 NAEP Mathematics Assessment. Author, Washington, D.C., 2020.
- OpenAI. text-embedding-3-large [Large language model], 2024.
- Vsurf: An r package for variable selection using random forests v 1.2.0. The R JOurnal, 7(2):19–33, 2015.
- G. Smith. Step away from stepwise. Journal of Big Data, 5(32):1–12, 2018.