Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
167 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

EduAgent: Generative Student Agents in Learning (2404.07963v1)

Published 23 Mar 2024 in cs.CY, cs.AI, cs.CL, cs.HC, and cs.LG

Abstract: Student simulation in online education is important to address dynamic learning behaviors of students with diverse backgrounds. Existing simulation models based on deep learning usually need massive training data, lacking prior knowledge in educational contexts. LLMs may contain such prior knowledge since they are pre-trained from a large corpus. However, because student behaviors are dynamic and multifaceted with individual differences, directly prompting LLMs is not robust nor accurate enough to capture fine-grained interactions among diverse student personas, learning behaviors, and learning outcomes. This work tackles this problem by presenting a newly annotated fine-grained large-scale dataset and proposing EduAgent, a novel generative agent framework incorporating cognitive prior knowledge (i.e., theoretical findings revealed in cognitive science) to guide LLMs to first reason correlations among various behaviors and then make simulations. Our two experiments show that EduAgent could not only mimic and predict learning behaviors of real students but also generate realistic learning behaviors of virtual students without real data.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (86)
  1. Gemini pro. https://ai.google.dev/tutorials/python_quickstart. Accessed: January 24, 2024.
  2. Openai. https://platform.openai.com/docs/models. Accessed: January 24, 2024.
  3. Llama 2 70b. https://huggingface.co/meta-llama/Llama-2-70b-chat-hf. Accessed: January 24, 2024.
  4. Statics2011 dataset. https://pslcdatashop.web.cmu.edu/DatasetInfo?datasetId=507. Accessed: January 24, 2024.
  5. Using large language models to simulate multiple humans and replicate human subject studies. In International Conference on Machine Learning, pages 337–371. PMLR, 2023.
  6. Autort: Embodied foundation models for large scale orchestration of robotic agents, 2024.
  7. Estimation of behavioral user state based on eye gaze and head pose—application in an e-learning environment. Multimedia Tools and Applications, 41:469–493, 2009.
  8. How to train your learners: Reinforcement learning for the scheduling of online learning activities. 2020.
  9. High-level student modeling with machine learning. In International Conference on Intelligent Tutoring Systems, pages 584–593. Springer, 2000.
  10. Aditi Bhutoria. Personalized education and artificial intelligence in the united states, china, and india: A systematic review using a human-in-the-loop model. Computers and Education: Artificial Intelligence, 3:100068, 2022.
  11. Mapping research in student engagement and educational technology in higher education: A systematic evidence map. International journal of educational technology in higher education, 17(1), 2020.
  12. Cognitive model priors for predicting human decisions. In International conference on machine learning, pages 5133–5141. PMLR, 2019.
  13. Jere E Brophy. Teacher behavior and student achievement. Number 73. Institute for Research on Teaching, Michigan State University, 1984.
  14. Online learning experiences of secondary school students during covid-19–dataset from vietnam. Data in Brief, 45:108662, 2022.
  15. Personality predicts academic performance: Evidence from two longitudinal university samples. Journal of research in personality, 37(4):319–338, 2003.
  16. Prerequisite-driven deep knowledge tracing. In 2018 IEEE International Conference on Data Mining (ICDM), pages 39–48. IEEE, 2018.
  17. Agentverse: Facilitating multi-agent collaboration and exploring emergent behaviors in agents. arXiv preprint arXiv:2308.10848, 2023.
  18. Ednet: A large-scale hierarchical dataset in education. In Artificial Intelligence in Education: 21st International Conference, AIED 2020, Ifrane, Morocco, July 6–10, 2020, Proceedings, Part II 21, pages 69–73. Springer, 2020.
  19. Samuel Rhys Cox. The use of multiple conversational agent interlocutors in learning. arXiv preprint arXiv:2312.16534, 2023.
  20. Affect and learning: an exploratory look into the role of affect in learning with autotutor. Journal of Educational Media, 29, 2004.
  21. Gary A Cziko. Unpredictability and indeterminism in human behavior: Arguments and implications for educational research. Educational researcher, 18(3):17–25, 1989.
  22. Student engagement dataset. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3628–3636, 2021.
  23. Mind2web: Towards a generalist agent for the web. arXiv preprint arXiv:2306.06070, 2023.
  24. Improved learning in a large-enrollment physics class. Science, 332(6031):862–864, 2011.
  25. The effects of flight complexity on gaze entropy: An experimental study with fighter pilots. Applied ergonomics, 77:92–99, 2019.
  26. Gaze tutor: A gaze-reactive intelligent tutoring system. International Journal of human-computer studies, 70(5):377–398, 2012.
  27. Yang Fan. Scb-dataset: A dataset for detecting student classroom behavior. arXiv preprint arXiv:2304.02488, 2023.
  28. Addressing the assessment challenge with an online system that tutors as it assesses. User modeling and user-adapted interaction, 19:243–266, 2009.
  29. Personality, cognitive ability, and beliefs about intelligence as predictors of academic performance. Learning and individual Differences, 14(1):47–64, 2003.
  30. S3: Social-network simulation system with large language model-empowered agents, 2023.
  31. Mindagent: Emergent gaming interaction. arXiv preprint arXiv:2309.09971, 2023.
  32. Information-seeking, curiosity, and attention: computational and neural mechanisms. Trends in cognitive sciences, 17(11):585–593, 2013.
  33. Qi Guo and Eugene Agichtein. Towards predicting web searcher gaze position from mouse movements. In CHI’10 extended abstracts on human factors in computing systems, pages 3601–3606. 2010.
  34. Safiyeh Rajaee Harandi. Effects of e-learning on students’ motivation. Procedia-Social and Behavioral Sciences, 181:423–430, 2015.
  35. Dataset of students’ performance using student information system, moodle and the mobile application “edify”. Data, 6(11):110, 2021.
  36. Using machine learning to predict student difficulties from learning session data. Artificial Intelligence Review, 52:381–407, 2019.
  37. Large language models in education: A focus on the complementary relationship between human teachers and chatgpt. Education and Information Technologies, pages 1–20, 2023.
  38. Lending interaction wings to recommender systems with conversational agents. arXiv preprint arXiv:2310.04230, 2023.
  39. Cgmi: Configurable general multi-agent interaction framework. arXiv preprint arXiv:2308.12503, 2023.
  40. The effects of academic environment and background characteristics on student satisfaction and performance: The case of south carolina state university’s school of business. College Student Journal, 37(2):298–309, 2003.
  41. Prediction and localization of student engagement in the wild. In 2018 Digital Image Computing: Techniques and Applications (DICTA), pages 1–8. IEEE, 2018.
  42. Identifying expert and novice visual scanpath patterns and their relationship to assessing learning-relevant student characteristics. In Frontiers in Education, volume 5, page 612175. Frontiers Media SA, 2021.
  43. Performance of chatgpt on usmle: Potential for ai-assisted medical education using large language models. PLoS digital health, 2(2):e0000198, 2023.
  44. Open university learning analytics dataset. Scientific data, 4(1):1–8, 2017.
  45. Cho Ki Kwok et al. Understanding user engagement level during tasks via facial responses, eye gaze and mouse movements. 2018.
  46. Cognitive abilities and motivational processes in high school students’ situational engagement and achievement in science. Educational Assessment, 8(2):139–162, 2002.
  47. Prediction of student performance in massive open online courses using deep learning system based on learning behaviors. Educational Technology & Society, 24(3):130–146, 2021.
  48. Relationships between student engagement and academic achievement: A meta-analysis. Social Behavior and Personality: an international journal, 46(3):517–528, 2018.
  49. Camel: Communicative agents for" mind" exploration of large language model society. In Thirty-seventh Conference on Neural Information Processing Systems, 2023.
  50. Swiftsage: A generative agent with fast and slow thinking for complex interactive tasks. arXiv preprint arXiv:2305.17390, 2023.
  51. Investigating the effect of an adaptive learning intervention on students’ learning. Educational technology research and development, 65:1605–1625, 2017.
  52. Ekt: Exercise-aware knowledge tracing for student performance prediction. IEEE Transactions on Knowledge and Data Engineering, 33(1):100–115, 2019.
  53. Xes3g5m: A knowledge tracing benchmark dataset with auxiliary information. In Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2023.
  54. Learning behaviours data in programming education: Community analysis and outcome prediction with cleaned data. Future Generation Computer Systems, 127:42–55, 2022.
  55. GPTeach: Interactive TA Training with GPT Based Students. 2023.
  56. Student activity and profile datasets from an online video-based collaborative learning experience. British Journal of Educational Technology, 46(5):993–998, 2015.
  57. When art moves the eyes: a behavioral and eye-tracking study. PloS one, 7(5):e37285, 2012.
  58. A large language model-assisted education tool to provide feedback on open-ended responses. arXiv preprint arXiv:2308.02439, 2023.
  59. Richard E. Mayer. Multimedia Learning. Cambridge University Press, 2nd edition, 2009. ISBN 0521514126.
  60. Catherine McLoughlin. Inclusivity and alignment: Principles of pedagogy, task and assessment design for effective cross-cultural online learning. Distance Education, 22(1):7–29, 2001.
  61. Deep knowledge tracing and dynamic student classification for knowledge tracing. In 2018 IEEE International conference on data mining (ICDM), pages 1182–1187. IEEE, 2018.
  62. Impact of learner’s characteristics and learning behaviour on learning performance during a fully online course. Note taking activities in e-learning environments, pages 15–36, 2021.
  63. Affective states and state tests: Investigating how affect throughout the school year predicts end of year learning outcomes. In Proceedings of the third international conference on learning analytics and knowledge, pages 117–124, 2013.
  64. Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology, pages 1–22, 2023.
  65. Deep knowledge tracing. Advances in neural information processing systems, 28, 2015.
  66. Junyi academy online learning activity dataset: A large-scale public online learning activity dataset from elementary to senior high school students. Dataset available from https://www.kaggle.com/junyiacademy/learning-activity-public-dataset-by-junyi-academy, 2020.
  67. Lauren B Resnick. Toward a cognitive theory of instruction. In Learning and motivation in the classroom, pages 5–38. Routledge, 2017.
  68. Atl-bp: a student engagement dataset and model for affect transfer learning for behavior prediction. IEEE Transactions on Biometrics, Behavior, and Identity Science, 2022.
  69. Student characteristics in the eyes of teachers: Differences between novice and expert teachers in judgment accuracy, observed behavioral cues, and gaze. Educational Psychology Review, 33:69–89, 2021.
  70. The 2010 kdd cup competition dataset: Engaging the machine learning community in predictive learning analytics. Journal of Learning Analytics, 3(2):312–316, 2016.
  71. Student class behavior dataset: a video dataset for recognizing, detecting, and captioning students’ behaviors in classroom scenes. Neural Computing and Applications, 33:8335–8354, 2021.
  72. Improving learning outcomes with gaze tracking and automatic question generation. In Proceedings of The Web Conference 2020, 2020.
  73. School practices to promote social distancing in k-12 schools: review of influenza pandemic policies and practices. BMC public health, 18(1):1–13, 2018.
  74. Predicting academic performance of students from vle big data using deep learning models. Computers in Human behavior, 104:106189, 2020.
  75. When large language model based agent meets user behavior analysis: A novel user simulation paradigm, 2023.
  76. Results and insights from diagnostic questions: The neurips 2020 education challenge. In NeurIPS 2020 Competition and Demonstration Track, pages 191–205. PMLR, 2021.
  77. Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, 35:24824–24837, 2022.
  78. Going deeper with deep knowledge tracing. International Educational Data Mining Society, 2016.
  79. A machine learning approach for tracking and predicting student performance in degree programs. IEEE Journal of Selected Topics in Signal Processing, 11(5):742–753, 2017.
  80. Leveraging generative artificial intelligence to simulate student learning behavior. arXiv preprint arXiv:2310.19206, 2023.
  81. Peer attention enhances student learning. arXiv e-prints, pages arXiv–2312, 2023.
  82. Webshop: Towards scalable real-world web interaction with grounded language agents. Advances in Neural Information Processing Systems, 35:20744–20757, 2022.
  83. On generative agents in recommendation. arXiv preprint arXiv:2310.10108, 2023.
  84. Data-driven online learning engagement detection via facial expression and mouse behavior recognition technology. Journal of Educational Computing Research, 58(1):63–86, 2020.
  85. End-to-end question generation to assist formative assessment design for conceptual knowledge learning. 2021.
  86. Integrating gaze and mouse via joint cross-attention fusion net for students’ activity recognition in e-learning. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 7(3):1–35, 2023.
Citations (1)

Summary

We haven't generated a summary for this paper yet.