Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

PhysicsAssistant: An LLM-Powered Interactive Learning Robot for Physics Lab Investigations (2403.18721v2)

Published 27 Mar 2024 in cs.RO

Abstract: Robot systems in education can leverage LLMs' (LLMs) natural language understanding capabilities to provide assistance and facilitate learning. This paper proposes a multimodal interactive robot (PhysicsAssistant) built on YOLOv8 object detection, cameras, speech recognition, and chatbot using LLM to provide assistance to students' physics labs. We conduct a user study on ten 8th-grade students to empirically evaluate the performance of PhysicsAssistant with a human expert. The Expert rates the assistants' responses to student queries on a 0-4 scale based on Bloom's taxonomy to provide educational support. We have compared the performance of PhysicsAssistant (YOLOv8+GPT-3.5-turbo) with GPT-4 and found that the human expert rating of both systems for factual understanding is the same. However, the rating of GPT-4 for conceptual and procedural knowledge (3 and 3.2 vs 2.2 and 2.6, respectively) is significantly higher than PhysicsAssistant (p < 0.05). However, the response time of GPT-4 is significantly higher than PhysicsAssistant (3.54 vs 1.64 sec, p < 0.05). Hence, despite the relatively lower response quality of PhysicsAssistant than GPT-4, it has shown potential for being used as a real-time lab assistant to provide timely responses and can offload teachers' labor to assist with repetitive tasks. To the best of our knowledge, this is the first attempt to build such an interactive multimodal robotic assistant for K-12 science (physics) education.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (31)
  1. I. Adeshola and A. P. Adepoju, “The opportunities and challenges of chatgpt in education,” Interactive Learning Environments, pp. 1–14, 2023.
  2. E. Latif, L. Fang, P. Ma, and X. Zhai, “Knowledge distillation of llm for education,” arXiv preprint arXiv:2312.15842, 2023.
  3. S. Vemprala, R. Bonatti, A. Bucker, and A. Kapoor, “Chatgpt for robotics: Design principles and model abilities,” Microsoft Auton. Syst. Robot. Res, vol. 2, p. 20, 2023.
  4. Y. Ye, H. You, and J. Du, “Improved trust in human-robot collaboration with chatgpt,” IEEE Access, 2023.
  5. E. Latif and X. Zhai, “Fine-tuning chatgpt for automatic scoring,” Computers and Education: Artificial Intelligence, p. 100210, 2024.
  6. Open AI, “Gpt-4v(ision) system card.” September 2023, published on September 25, 2023.
  7. ——, “Chatgpt can now see, hear, and speak,” https://openai.com/blog/chatgpt-can-now-see-hear-and-speak, September 2023, published on September 25, 2023.
  8. G.-G. Lee, E. Latif, L. Shi, and X. Zhai, “Gemini pro defeated by gpt-4v: Evidence from education,” arXiv preprint arXiv:2401.08660, 2023.
  9. R. Yang, L. Song, Y. Li, S. Zhao, Y. Ge, X. Li, and Y. Shan, “Gpt4tools: Teaching large language model to use tools via self-instruction,” Advances in Neural Information Processing Systems, vol. 36, 2024.
  10. G.-G. Lee, L. Shi, E. Latif, Y. Gao, A. Bewersdorf, M. Nyaaba, S. Guo, Z. Wu, Z. Liu, H. Wang et al., “Multimodality of ai for education: Towards artificial general intelligence,” arXiv preprint arXiv:2312.06037, 2023.
  11. H. Lou, X. Duan, J. Guo, H. Liu, J. Gu, L. Bi, and H. Chen, “Dc-yolov8: Small-size object detection algorithm based on camera sensor,” Electronics, vol. 12, no. 10, p. 2323, 2023.
  12. Z. Zang, C. Lin, C. Tang, T. Wang, and J. Lv, “Zero-shot aerial object detection with visual description regularization,” arXiv preprint arXiv:2402.18233, 2024.
  13. Y. Jin, Z. Shi, X. Xu, G. Wu, H. Li, and S. Wen, “Target localization and grasping of nao robot based on yolov8 network and monocular ranging,” Electronics, vol. 12, no. 18, p. 3981, 2023.
  14. L. Shen, B. Lang, and Z. Song, “Ds-yolov8-based object detection method for remote sensing images,” IEEE Access, vol. 11, pp. 125 122–125 137, 2023.
  15. A. Koubaa, “Rosgpt: Next-generation human-robot interaction with chatgpt and ros,” 2023.
  16. C. Y. Kim, C. P. Lee, and B. Mutlu, “Understanding large-language model (llm)-powered human-robot interaction,” arXiv preprint arXiv:2401.03217, 2024.
  17. S. Gupta and Y. Chen, “Supporting inclusive learning using chatbots? a chatbot-led interview study,” Journal of Information Systems Education, vol. 33, no. 1, pp. 98–108, 2022.
  18. H. Yu, “The application and challenges of chatgpt in educational transformation: New demands for teachers’ roles,” Heliyon, 2024.
  19. S. Biswas, “Role of chat gpt in education,” Available at SSRN 4369981, 2023.
  20. K. Jangjarat, T. Kraiwanit, P. Limna, and R. Sonsuphap, “Public perceptions towards chatgpt as the robo-assistant,” Jangjarat, K., Kraiwanit, T., Limna, P., & Sonsuphap, 2023.
  21. A. Shafeeg, I. Shazhaev, D. Mihaylov, A. Tularov, and I. Shazhaev, “Voice assistant integrated with chat gpt,” Indonesian Journal of Computer Science, vol. 12, no. 1, 2023.
  22. E. Latif, G.-G. Lee, K. Neuman, T. Kastorff, and X. Zhai, “G-sciedbert: A contextualized llm for science assessment tasks in german,” arXiv preprint arXiv:2402.06584, 2024.
  23. M. T. Chandio, S. M. Pandhiani, and R. Iqbal, “Bloom’s taxonomy: Improving assessment and teaching-learning process.” Journal of education and educational development, vol. 3, no. 2, pp. 203–221, 2016.
  24. G. Wilcock and K. Jokinen, “To err is robotic; to earn trust, divine: Comparing chatgpt and knowledge graphs for hri,” in 2023 32nd IEEE International Conference on Robot and Human Interactive Communication (RO-MAN).   IEEE, 2023, pp. 1396–1401.
  25. M. Donnermann, P. Schaper, and B. Lugrin, “Investigating adaptive robot tutoring in a long-term interaction in higher education,” in 2022 31st IEEE International Conference on Robot and Human Interactive Communication (RO-MAN).   IEEE, 2022, pp. 171–178.
  26. I. Cucciniello, G. L’Arco, A. Rossi, C. Autorino, G. Santoro, and S. Rossi, “Classmate robot: A robot to support teaching and learning activities in schools,” in 2022 31st IEEE International Conference on Robot and Human Interactive Communication (RO-MAN).   IEEE, 2022, pp. 906–911.
  27. A.-M. Velentza, N. Fachantidis, and I. Lefkos, “Human or robot university tutor? future teachers’ attitudes and learning outcomes,” in 2021 30th IEEE International Conference on Robot & Human Interactive Communication (RO-MAN).   IEEE, 2021, pp. 236–242.
  28. N. Li, Y. Liu, Y. Wu, S. Liu, S. Zhao, and M. Liu, “Robutrans: A robust transformer-based text-to-speech model,” in Proceedings of the AAAI conference on artificial intelligence, vol. 34, no. 05, 2020, pp. 8228–8235.
  29. K. L. Ritchie, “Using irb protocols to teach ethical principles for research and everyday life: A high-impact practice.” Journal of the Scholarship of Teaching and Learning, vol. 21, no. 1, pp. 120–130, 2021.
  30. G.-G. Lee, E. Latif, X. Wu, N. Liu, and X. Zhai, “Applying large language models and chain-of-thought for automatic scoring,” Computers and Education: Artificial Intelligence, p. 100213, 2024.
  31. J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat et al., “Gpt-4 technical report,” arXiv preprint arXiv:2303.08774, 2023.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (3)
  1. Ehsan Latif (36 papers)
  2. Ramviyas Parasuraman (51 papers)
  3. Xiaoming Zhai (48 papers)
Citations (5)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now