Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
125 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

VIVID: Human-AI Collaborative Authoring of Vicarious Dialogues from Lecture Videos (2403.09168v2)

Published 14 Mar 2024 in cs.HC

Abstract: The lengthy monologue-style online lectures cause learners to lose engagement easily. Designing lectures in a "vicarious dialogue" format can foster learners' cognitive activities more than monologue-style. However, designing online lectures in a dialogue style catered to the diverse needs of learners is laborious for instructors. We conducted a design workshop with eight educational experts and seven instructors to present key guidelines and the potential use of LLMs (LLM) to transform a monologue lecture script into pedagogically meaningful dialogue. Applying these design guidelines, we created VIVID which allows instructors to collaborate with LLMs to design, evaluate, and modify pedagogical dialogues. In a within-subjects study with instructors (N=12), we show that VIVID helped instructors select and revise dialogues efficiently, thereby supporting the authoring of quality dialogues. Our findings demonstrate the potential of LLMs to assist instructors with creating high-quality educational dialogues across various learning stages.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (77)
  1. Intelligent tutoring systems. Science 228, 4698 (1985), 456–462.
  2. Albert Bandura and Richard H Walters. 1977. Social learning theory. Vol. 1. Englewood cliffs Prentice Hall.
  3. Garima Bansal. 2018. Teacher discursive moves: Conceptualising a schema of dialogic discourse in science classrooms. International Journal of Science Education 40, 15 (2018), 1891–1912.
  4. Characteristics of instructional videos. World Journal on Educational Technology: Current Issues 10, 1 (2018), 61–69.
  5. Learner characteristics and feedback in tutorial dialogue. In Proceedings of the Third Workshop on Innovative Use of NLP for Building Educational Applications. 53–61.
  6. Discovering Tutorial Dialogue Strategies with Hidden Markov Models.. In AIED. 141–148.
  7. Balancing cognitive and motivational scaffolding in tutorial dialogue. In Intelligent Tutoring Systems: 9th International Conference, ITS 2008, Montreal, Canada, June 23-27, 2008 Proceedings 9. Springer, 239–249.
  8. H David Brecht. 2012. Learning from online video lectures. Journal of Information Technology Education. Innovations in Practice 11 (2012), 227.
  9. Karin Brodie. 2011. Working with learners’ mathematical thinking: Towards a language of description for changing pedagogy. Teaching and Teacher Education 27, 1 (2011), 174–186.
  10. Dialogue modes in expert tutoring. In Intelligent Tutoring Systems: 9th International Conference, ITS 2008, Montreal, Canada, June 23-27, 2008 Proceedings 9. Springer, 470–479.
  11. Why Students Learn More From Dialogue- Than Monologue-Videos: Analyses of Peer Interactions. Journal of the Learning Sciences 26 (06 2016). https://doi.org/10.1080/10508406.2016.1204546
  12. Observing Tutorial Dialogues Collaboratively: Insights About Human Tutoring Effectiveness From Vicarious Learning. Cognitive science 32 (03 2008), 301–41. https://doi.org/10.1080/03640210701863396
  13. Learning from human tutoring. Cognitive science 25, 4 (2001), 471–533.
  14. Christine Chin. 2006. Classroom interaction in science: Teacher questioning and feedback to students’ responses. International journal of science education 28, 11 (2006), 1315–1346.
  15. Konstantinos Chorianopoulos. 2018. A taxonomy of asynchronous instructional video styles. The International Review of Research in Open and Distributed Learning 19, 1 (2 2018). https://doi.org/10.19173/irrodl.v19i1.2920
  16. Promoting vicarious learning of physics using deep questions with explanations. Computers & Education 58, 4 (2012), 1042–1048.
  17. The deep-level-reasoning-question effect: The role of dialogue and deep-level-reasoning questions during vicarious learning. Cognition and Instruction 24, 4 (2006), 565–591.
  18. Human vs Machine: Comparison of Student-generated and AI-generated Educational Content. arXiv preprint arXiv:2306.10509 (2023).
  19. Learning from error episodes in dialogue-videos: The influence of prior knowledge. Australasian Journal of Educational Technology (03 2021), 20–32. https://doi.org/10.14742/ajet.6239
  20. Expert tutors feedback is immediate, direct, and discriminating. In Twenty-Third International FLAIRS Conference.
  21. Vicarious learning: Effects of overhearing dialog and monologue-like discourse in a virtual tutoring session. Journal of Educational Computing Research 29, 4 (2003), 431–450.
  22. Ilana Dubovi and Victor R Lee. 2019. Instructional support for learning with agent-based simulations: A tale of vicarious and guided exploration learning approaches. Computers & Education 142 (2019), 103644.
  23. Matthew S Ellman and Michael L Schwartz. 2016. Article commentary: Online learning tools as supplements for basic and clinical science education. Journal of medical education and curricular development 3 (2016), JMECD–S18933.
  24. Iris: A conversational agent for complex tasks. In Proceedings of the 2018 CHI conference on human factors in computing systems. 1–12.
  25. Mary Forehand. 2010. Bloom’s taxonomy. Emerging perspectives on learning, teaching, and technology 41, 4 (2010), 47–56.
  26. Exploring the deep-level reasoning questions effect during vicarious learning among eighth to eleventh graders in the domains of computer literacy and Newtonian physics. Instructional Science 37 (2009), 487–493.
  27. Arthur C Graesser and Sidney D’Mello. 2012. Emotions during the learning of difficult material. In Psychology of learning and motivation. Vol. 57. Elsevier, 183–225.
  28. AutoTutor: A tutor with dialogue in natural language. Behavior Research Methods, Instruments, & Computers 36 (2004), 180–192.
  29. AutoTutor: A simulation of a human tutor. Cognitive Systems Research 1, 1 (1999), 35–51.
  30. MathBot: Transforming online resources for learning math into conversational interactions. AAAI 2019 Story-Enabled Intelligence (2019).
  31. Hinting as a tactic in one-on-one tutoring. The Journal of the Learning Sciences 5, 1 (1996), 23–47.
  32. Stanley D Ivie. 1998. Ausubel’s learning theory: An approach to teaching higher order thinking skills. The High School Journal 82, 1 (1998), 35–42.
  33. The Classroom Discourse Observation Protocol (CDOP): A quantitative method for characterizing teacher discourse moves in undergraduate STEM learning environments. PloS one 14, 7 (2019), e0219019.
  34. James A Kulik and JD Fletcher. 2016. Effectiveness of intelligent tutoring systems: a meta-analytic review. Review of educational research 86, 1 (2016), 42–78.
  35. Development of an instructional design model for flipped learning in higher education. Educational Technology Research and Development 65 (2017), 427–453.
  36. Curiosity notebook: the design of a research platform for learning by teaching. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–26.
  37. Promptiverse: Scalable generation of scaffolding prompts through human-AI hybrid knowledge graph annotation. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. 1–18.
  38. Confusion and complex learning during interactions with computer learning environments. The Internet and Higher Education 15, 3 (2012), 184–194.
  39. Comparing code explanations created by students and large language models. arXiv preprint arXiv:2304.03938 (2023).
  40. Examining studies of inquiry-based learning in three fields of education: Sparking generative conversation. Journal of teacher education 64, 5 (2013), 387–408.
  41. Diane Litman and Kate Forbes-Riley. 2006. Correlations between dialogue acts and learning in spoken tutoring dialogues. Natural Language Engineering 12, 2 (2006), 161–176.
  42. Thomas Lord and Sandhya Baviskar. 2007. Moving students from information recitation to information understanding-Exploiting Bloom’s Taxonomy in creating science questions. Journal of College Science Teaching 36, 5 (2007), 40.
  43. Expert vs. non-expert tutoring: Dialogue moves, interaction patterns and multi-utterance turns. In Computational Linguistics and Intelligent Text Processing: 8th International Conference, CICLing 2007, Mexico City, Mexico, February 18-24, 2007. Proceedings 8. Springer, 456–467.
  44. ReadingQuizMaker: A Human-NLP Collaborative System that Supports Instructors to Design High-Quality Reading Quiz Questions. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–18.
  45. Sue Lyle. 2008. Dialogic teaching: Discussing theoretical contexts and reviewing evidence from classroom practice. Language and education 22, 3 (2008), 222–240.
  46. Intelligent tutoring systems and learning outcomes: A meta-analysis. Journal of educational psychology 106, 4 (2014), 901.
  47. MTFeedback: providing notifications to enhance teacher awareness of small group work in the classroom. IEEE Transactions on Learning Technologies 8, 2 (2014), 187–200.
  48. J Terry Mayes. 2015. Still to learn from vicarious learning. E-learning and digital media 12, 3-4 (2015), 361–371.
  49. The effect of positive feedback in a constraint-based intelligent tutoring system. Computers & Education 60, 1 (2013), 264–272.
  50. Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA statement. Br Med J 8 (07 2009), 336–341. https://doi.org/10.1371/joumal.pmedl000097
  51. Quality of educational dialogue and association with students’ academic performance. Learning and Instruction 55 (2018), 67–79.
  52. Scaffolding through dialogic teaching in early school classrooms. Teaching and teacher education 55 (2016), 143–154.
  53. Learning by observing tutorial dialogue versus monologue collaboratively or alone. Proceedings of the 33rd Annual Conference of the Cognitive Science Society.
  54. A Tool to Add a Tutee Agent in a Monologue Lecture Video Improves Students’ Watching Experience. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems. 1–7.
  55. Promoting higher-order thinking through teacher questioning: a case study of a Singapore science classroom. New Waves-Educational Research and Development Journal 19, 1 (2016), 1–19.
  56. José Paladines and Jaime Ramirez. 2020. A systematic literature review of intelligent tutoring systems with dialogue in natural language. IEEE Access 8 (2020), 164246–164267.
  57. Robust speech recognition via large-scale weak supervision. In International Conference on Machine Learning. PMLR, 28492–28518.
  58. The role of why questions in effective human tutoring. In Proceedings of AIED, Vol. 3.
  59. Quizbot: A dialogue-based adaptive learning system for factual knowledge. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–13.
  60. Automatic generation of programming exercises and code explanations using large language models. In Proceedings of the 2022 ACM Conference on International Computing Education Research-Volume 1. 27–43.
  61. Harnessing llms in curricular design: Using gpt-4 to support authoring of learning objectives. arXiv preprint arXiv:2306.17459 (2023).
  62. MAJ Stanislaw Paul. 2021. World Class STEM - Benchmarking and Delivering based on Evidence Based Cognitive Science. In 2021 IEEE International Conference on Engineering, Technology & Education (TALE). 1139–1144. https://doi.org/10.1109/TALE52509.2021.9678899
  63. Suhirman Suhirman and Saiful Prayogi. 2023. Overcoming challenges in STEM education: A literature review that leads to effective pedagogy in STEM learning. Jurnal Penelitian Pendidikan IPA 9, 8 (2023), 432–443.
  64. Leah A Sutton. 2001. The principle of vicarious interaction in computer-mediated communications. International Journal of Educational Telecommunications 7, 3 (2001), 223–242.
  65. Karen Swan. 2003. Learning effectiveness online: What the research tells us. Elements of quality online education, practice and direction 4, 1 (2003), 13–47.
  66. Authoring Virtual Peer Interactions for Lecture Videos. In CHI Conference on Human Factors in Computing Systems Extended Abstracts. 1–7.
  67. Scripted Vicarious Dialogues: Educational Video Augmentation Method for Increasing Isolated Students’ Engagement. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–25.
  68. Peter Teo. 2016. Exploring the dialogic space in teaching: A study of teacher talk in the pre-university classroom in Singapore. Teaching and Teacher Education 56 (2016), 47–60.
  69. Why do only some events cause learning during human tutoring? Cognition and Instruction 21, 3 (2003), 209–249.
  70. Dialogic practices in primary school classrooms. Language and Education 33, 1 (2019), 85–100.
  71. ArgueTutor: An adaptive dialog-based learning system for argumentation skills. In Proceedings of the 2021 CHI conference on human factors in computing systems. 1–13.
  72. Seeing beyond expert blind spots: Online learning design for scale and quality. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–14.
  73. QG-net: a data-driven question generation model for educational content. In Proceedings of the fifth annual ACM conference on learning at scale. 1–10.
  74. Key phrase extraction for generating educational question-answer pairs. In Proceedings of the Sixth (2019) ACM Conference on Learning@ Scale. 1–10.
  75. Sara, the lecturer: Improving learning in online education with a scaffolding-based conversational agent. In Proceedings of the 2020 CHI conference on human factors in computing systems. 1–14.
  76. Cognitive mirage: A review of hallucinations in large language models. arXiv preprint arXiv:2309.06794 (2023).
  77. Siren’s song in the AI ocean: a survey on hallucination in large language models. arXiv preprint arXiv:2309.01219 (2023).
Citations (4)

Summary

We haven't generated a summary for this paper yet.