Chatbot Conversations in Physics Education: Using Artificial Intelligence to Analyze Student Reasoning through Computational Grounded Theory

Abstract: This study applies Computational Grounded Theory (CGT) to analyze student misconceptions using interaction data from an AI-powered chatbot deployed in a university-level Modern Physics course. The chatbot - the UTA Study Buddy Bot - engaged students in peer-like problem-solving conversations throughout the semester, generating a rich dataset of over 10 million tokens. To explore patterns in student reasoning and identify recurring conceptual difficulties, we implemented a CGT pipeline that combined natural language processing, unsupervised clustering of sentence-level vector embeddings, human interpretation of emergent themes, and supervised learning to evaluate the generalizability of identified categories. Preliminary results revealed persistent misconceptions in areas such as relativistic momentum and quantum energy levels, along with distinctive trends in how students phrased their questions and expressed uncertainty. These findings underscore the potential of CGT as a scalable, theory-aligned approach for extracting insights from chatbot dialogues and guiding the development of more adaptive, AI-driven educational tools in physics instruction.