Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
41 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
41 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Book2Dial: Generating Teacher-Student Interactions from Textbooks for Cost-Effective Development of Educational Chatbots (2403.03307v1)

Published 5 Mar 2024 in cs.CL

Abstract: Educational chatbots are a promising tool for assisting student learning. However, the development of effective chatbots in education has been challenging, as high-quality data is seldom available in this domain. In this paper, we propose a framework for generating synthetic teacher-student interactions grounded in a set of textbooks. Our approaches capture one aspect of learning interactions where curious students with partial knowledge interactively ask a teacher questions about the material in the textbook. We highlight various quality criteria that such dialogues should fulfill and compare several approaches relying on either prompting or fine-tuning LLMs. We use synthetic dialogues to train educational chatbots and show benefits of further fine-tuning in different educational domains. However, human evaluation shows that our best data synthesis method still suffers from hallucinations and tends to reiterate information from previous conversations. Our findings offer insights for future efforts in synthesizing conversational data that strikes a balance between size and quality. We will open-source our data and code.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (55)
  1. Open-domain question answering goes conversational via question rewriting. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 520–534, Online. Association for Computational Linguistics.
  2. Educational dialogues and computer supported collaborative learning: critical analysis and research perspectives. International Journal of Computer-Supported Collaborative Learning, pages 1–22.
  3. A synthetic data generation framework for grounded dialogues. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 10866–10882, Toronto, Canada. Association for Computational Linguistics.
  4. Taskmaster-2. https://github.com/google-research-datasets/Taskmaster/tree/master/TM-2-2020. Second dataset in series of three.
  5. The teacher-student chatroom corpus. In Proceedings of the 9th Workshop on NLP for Computer Assisted Language Learning, pages 10–20.
  6. PLACES: Prompting language models for social conversation synthesis. In Findings of the Association for Computational Linguistics: EACL 2023, pages 844–868, Dubrovnik, Croatia. Association for Computational Linguistics.
  7. Michelene TH Chi and Ruth Wylie. 2014. The icap framework: Linking cognitive engagement to active learning outcomes. Educational psychologist, 49(4):219–243.
  8. QuAC: Question answering in context. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2174–2184, Brussels, Belgium. Association for Computational Linguistics.
  9. Scaling instruction-finetuned language models. arXiv preprint arXiv:2210.11416.
  10. Elastic weight removal for faithful and abstractive dialogue generation. arXiv preprint arXiv:2303.17574.
  11. Dialogue response generation via contrastive latent representation learning. In Proceedings of the 3rd Workshop on Natural Language Processing for Conversational AI, pages 189–197, Online. Association for Computational Linguistics.
  12. Dialog inpainting: Turning documents into dialogs. In International Conference on Machine Learning, pages 4558–4586. PMLR.
  13. Dorottya Demszky and Heather Hill. 2023. The NCTE transcripts: A dataset of elementary math classroom transcripts. In Proceedings of the 18th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2023), pages 528–538, Toronto, Canada. Association for Computational Linguistics.
  14. Measuring conversational uptake: A case study on student-teacher interactions. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 1638–1653, Online. Association for Computational Linguistics.
  15. Modeling what-to-ask and how-to-ask for answer-unaware conversational question generation. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 10785–10803, Toronto, Canada. Association for Computational Linguistics.
  16. Evaluating coherence in dialogue systems using entailment. In Proceedings of the 2019 Workshop on Widening NLP, pages 146–148, Florence, Italy. Association for Computational Linguistics.
  17. Active learning increases student performance in science, engineering, and mathematics. Proceedings of the national academy of sciences, 111(23):8410–8415.
  18. Jonathan Ginzburg. 2010. Relevance for dialogue. In SemDial: Workshop on the Semantics and Pragmatics of Dialogue (PozDial), pages 121–129.
  19. Arthur C Graesser and Natalie K Person. 1994. Question asking during tutoring. American educational research journal, 31(1):104–137.
  20. Newsroom: A dataset of 1.3 million summaries with diverse extractive strategies. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 708–719, New Orleans, Louisiana. Association for Computational Linguistics.
  21. q2superscript𝑞2q^{2}italic_q start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT: Evaluating factual consistency in knowledge-grounded dialogues via question generation and question answering. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 7856–7870, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  22. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations.
  23. Dialogizer: Context-aware conversational-QA dataset generation from textual sources. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 8806–8828, Singapore. Association for Computational Linguistics.
  24. Survey of hallucination in natural language generation. ACM Computing Surveys, 55(12):1–38.
  25. Generating information-seeking conversations from unlabeled documents. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 2362–2378.
  26. SODA: Million-scale dialogue distillation with social commonsense contextualization. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 12930–12949, Singapore. Association for Computational Linguistics.
  27. Interacting with educational chatbots: A systematic review. Education and Information Technologies, 28(1):973–1018.
  28. A systematic review of automatic question generation for educational purposes. International Journal of Artificial Intelligence in Education, 30:121–204.
  29. Mark R. Lepper and Maria Woolverton. 2002. Chapter 7 - the wisdom of practice: Lessons learned from the study of highly effective tutors. In Joshua Aronson, editor, Improving Academic Achievement, Educational Psychology, pages 135–158. Academic Press, San Diego.
  30. I Loshchilov and F Hutter. 2019. " decoupled weight decay regularization", 7th international conference on learning representations, iclr. New Orleans, LA, USA, May, (6-9):2019.
  31. MathDial: A dialogue tutoring dataset with rich pedagogical properties grounded in math reasoning problems. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 5602–5621, Singapore. Association for Computational Linguistics.
  32. Opportunities and challenges in neural dialog tutoring. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 2357–2372, Dubrovnik, Croatia. Association for Computational Linguistics.
  33. Gpteach: Interactive ta training with gpt based students.
  34. Richard E Mayer and Roxana Moreno. 2003. Nine ways to reduce cognitive load in multimedia learning. Educational psychologist, 38(1):43–52.
  35. College student web use, perceptions of information credibility, and verification behavior. Computers & Education, 41(3):271–290.
  36. RQUGE: Reference-free metric for evaluating question generation by answering the question. In Findings of the Association for Computational Linguistics: ACL 2023, pages 6845–6867, Toronto, Canada. Association for Computational Linguistics.
  37. Inferring what the student knows in one-to-one tutoring: The role of student questions and answers. Learning and individual differences, 6(2):205–229.
  38. Matt Post. 2018. A call for clarity in reporting BLEU scores. In Proceedings of the Third Conference on Machine Translation: Research Papers, pages 186–191, Brussels, Belgium. Association for Computational Linguistics.
  39. Stay hungry, stay focused: Generating informative and specific questions in information-seeking conversations. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 25–40, Online. Association for Computational Linguistics.
  40. Open-retrieval conversational question answering. In Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval, pages 539–548.
  41. Educational multi-question generation for reading comprehension. In Proceedings of the 17th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2022), pages 216–223.
  42. Coqa: A conversational question answering challenge. Transactions of the Association for Computational Linguistics, 7:249–266.
  43. Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence embeddings using Siamese BERT-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982–3992, Hong Kong, China. Association for Computational Linguistics.
  44. Quizbot: A dialogue-based adaptive learning system for factual knowledge. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, pages 1–13.
  45. QuestEval: Summarization asks for fact-based evaluation. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6594–6604, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  46. Thomas Scialom and Jacopo Staiano. 2020. Ask to learn: A study on curiosity-driven question generation. In Proceedings of the 28th International Conference on Computational Linguistics, pages 2224–2235, Barcelona, Spain (Online). International Committee on Computational Linguistics.
  47. Automatic generation of socratic subquestions for teaching math word problems. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 4136–4149, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  48. CLASS: A design framework for building intelligent tutoring systems based on learning science principles. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 1941–1961, Singapore. Association for Computational Linguistics.
  49. Cima: A large open access dialogue dataset for tutoring. In Proceedings of the Fifteenth Workshop on Innovative Use of NLP for Building Educational Applications, pages 52–64.
  50. Automatically generating cause-and-effect questions from passages. In Proceedings of the 16th Workshop on Innovative Use of NLP for Building Educational Applications, pages 158–170.
  51. Fine-tuning transformers with additional context to classify discursive moves in mathematics classrooms. In Proceedings of the 17th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2022), pages 71–81, Seattle, Washington. Association for Computational Linguistics.
  52. Does informativeness matter? active learning for educational dialogue act classification. In International Conference on Artificial Intelligence in Education, pages 176–188. Springer.
  53. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 conference on empirical methods in natural language processing: system demonstrations, pages 38–45.
  54. Synthetic data generation for end-to-end thermal infrared tracking. IEEE Transactions on Image Processing, 28(4):1837–1850.
  55. Bertscore: Evaluating text generation with bert. In International Conference on Learning Representations.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (5)
  1. Junling Wang (21 papers)
  2. Jakub Macina (9 papers)
  3. Nico Daheim (24 papers)
  4. Sankalan Pal Chowdhury (9 papers)
  5. Mrinmaya Sachan (124 papers)
Citations (4)
View on Semantic Scholar
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets