Automated Generation of Multiple-Choice Cloze Questions for Assessing English Vocabulary Using GPT-turbo 3.5 (2403.02078v1)
Abstract: A common way of assessing language learners' mastery of vocabulary is via multiple-choice cloze (i.e., fill-in-the-blank) questions. But the creation of test items can be laborious for individual teachers or in large-scale language programs. In this paper, we evaluate a new method for automatically generating these types of questions using LLMs (LLM). The VocaTT (vocabulary teaching and training) engine is written in Python and comprises three basic steps: pre-processing target word lists, generating sentences and candidate word options using GPT, and finally selecting suitable word options. To test the efficiency of this system, 60 questions were generated targeting academic words. The generated items were reviewed by expert reviewers who judged the well-formedness of the sentences and word options, adding comments to items judged not well-formed. Results showed a 75% rate of well-formedness for sentences and 66.85% rate for suitable word options. This is a marked improvement over the generator used earlier in our research which did not take advantage of GPT's capabilities. Post-hoc qualitative analysis reveals several points for improvement in future work including cross-referencing part-of-speech tagging, better sentence validation, and improving GPT prompts.
- Chatgpt: Fundamentals, applications and social impacts. In 2022 Ninth International Conference on Social Networks Analysis, Management and Security (SNAMS), pages 1–8. IEEE.
- Mofareh Alqahtani et al. 2015. The importance of vocabulary in language learning and how to be taught. International journal of teaching and education, 3(3):21–34.
- Automatic question generation for vocabulary assessment. In Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing, pages 819–826.
- David Coniam. 1997. A computerised english language proofing cloze program. Computer Assisted Language Learning, 10(1):83–97.
- Averil Coxhead. 2000. A new academic word list. TESOL quarterly, 34(2):213–238.
- Interrater agreement and interrater reliability: key concepts, approaches, and applications. Research in Social and Administrative Pharmacy, 9(3):330–338.
- Arthur C Graesser. 2001. Question generation as a learning multiplier in distributed learning environments, volume 1121. US Army Research Institute for the Behavioral and Social Sciences.
- The relation of multiple-choice cloze items to the test of english as a foreign language. Language Testing, 6(1):47–76.
- Filtering-based automatic cloze test generation. In Proceedings of Speech and Language Technology in Education (SLaTE), page 72–76.
- Applications of lexical information for algorithmically composing multiple-choice cloze items. In Proceedings of the 2nd Workshop on Building Educational Applications Using NLP, page 1–8.
- Paul Nation. 2022. Teaching and learning vocabulary. In Eli Hinkel, editor, Handbook of Practical Second Language Teaching and Learning, pages 397–408. Routledge, New York, NY, USA.
- A selection strategy to improve cloze question quality. In Proceedings of the Workshop on Intelligent Tutoring Systems for Ill-Defined Domains. 9th International Conference on Intelligent Tutoring Systems, Montreal, Canada, pages 22–32.
- Ralph L. Rose. 2014. Automated vocabulary quiz creation using online and offline corpora. Poster presentation at Teaching and Language Corpora (TaLC) conference, Lancaster University, UK.
- Ralph L. Rose. 2016. Automatic word quiz construction using regular and simple english wikipedia. In INTED2016 Proceedings, pages 8032–8040. IATED.
- Ralph L. Rose. 2020. Improving the production efficiency and well-formedness of automatically-generated multiple-choice cloze vocabulary questions. In Proceedings of the 12th Language Resources and Evaluation Conference, pages 7094–7101.
- Evaluation dataset of multiple-choice cloze items for vocabulary training and testing. In Adjunct Proceedings of the 2022 ACM International Joint Conference on Pervasive and Ubiquitous Computing and the 2022 ACM International Symposium on Wearable Computers, pages 292–298.
- Norbert Schmitt. 1997. Vocabulary learning strategies. In Norbert Schmitt and Michael McCarthy, editors, Vocabulary: Description, acquisition and pedagogy, pages 199–227. Cambridge University Press, Cambridge, UK.
- Cheryl Boyd Zimmerman. 1997. Do reading and interactive vocabulary instruction make a difference? an empirical study. TESOL quarterly, 31(1):121–140.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.