Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
144 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

FOLLOWUPQG: Towards Information-Seeking Follow-up Question Generation (2309.05007v2)

Published 10 Sep 2023 in cs.CL and cs.AI

Abstract: Humans ask follow-up questions driven by curiosity, which reflects a creative human cognitive process. We introduce the task of real-world information-seeking follow-up question generation (FQG), which aims to generate follow-up questions seeking a more in-depth understanding of an initial question and answer. We construct FOLLOWUPQG, a dataset of over 3K real-world (initial question, answer, follow-up question) tuples collected from a Reddit forum providing layman-friendly explanations for open-ended questions. In contrast to existing datasets, questions in FOLLOWUPQG use more diverse pragmatic strategies to seek information, and they also show higher-order cognitive skills (such as applying and relating). We evaluate current question generation models on their efficacy for generating follow-up questions, exploring how to generate specific types of follow-up questions based on step-by-step demonstrations. Our results validate FOLLOWUPQG as a challenging benchmark, as model-generated questions are adequate but far from human-raised questions in terms of informativeness and complexity.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (42)
  1. A taxonomy for learning, teaching, and assessing: A revision of bloom’s taxonomy of educational objectives, abridged edition. White Plains, NY: Longman.
  2. GPT-Neo: Large Scale Autoregressive Language Modeling with Mesh-Tensorflow.
  3. Learningq: A large-scale dataset for educational question generation. In International Conference on Web and Social Media (ICWSM), pages 481–490.
  4. JT Dillon. 1988. Questioning and teaching. a manual of practice.
  5. Xinya Du and Claire Cardie. 2018. Harvesting paragraph-level question-answer pairs from wikipedia. In Annual Meeting of the Association for Computational Linguistics (ACL), pages 1907–1917.
  6. Learning to ask: Neural question generation for reading comprehension. In Annual Meeting of the Association for Computational Linguistics (ACL), pages 1342–1352.
  7. Question generation for question answering. In Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 866–874.
  8. ELI5: long form question answering. In Annual Meeting of the Association for Computational Linguistics (ACL), pages 3558–3567.
  9. Question generation for reading comprehension assessment by modeling how and what to ask. In Findings of Annual Meeting of the Association for Computational Linguistics (ACL), pages 2131–2146.
  10. Arthur C Graesser and Natalie K Person. 1994. Question asking during tutoring. American educational research journal, 31(1):104–137.
  11. Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings.
  12. Inquisitive question generation for high level text comprehension. In Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6544–6555.
  13. Vaibhav Kumar and Alan W. Black. 2020. Clarq: A large-scale and diverse dataset for clarification question generation. In Annual Meeting of the Association for Computational Linguistics (ACL), pages 7296–7301.
  14. Quiz design task: Helping teachers create quizzes with automated question generation. CoRR, abs/2205.01730.
  15. Alon Lavie and Abhaya Agarwal. 2007. METEOR: An automatic metric for MT evaluation with high levels of correlation with human judgments. In Proceedings of the Second Workshop on Statistical Machine Translation, pages 228–231.
  16. BART: denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In Annual Meeting of the Association for Computational Linguistics (ACL), pages 7871–7880.
  17. Chin-Yew Lin. 2004. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out, pages 74–81.
  18. Learning to generate questions by learning what not to generate. In International World Wide Web Conference (WWW), pages 1106–1118.
  19. Ask what’s missing and what’s useful: Improving clarification question generation using global knowledge. In Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL-HLT), pages 4300–4312.
  20. Christopher Malon and Bing Bai. 2020. Generating followup questions for interpretable multi-hop question answering. ArXiv, abs/2002.12344.
  21. Naomi Miyake and Donald A Norman. 1979. To ask a question, one must know enough to know what is not known. Journal of verbal learning and verbal behavior, 18(3):357–364.
  22. Mixqg: Neural question generation with mixed answer types. CoRR, abs/2110.08175.
  23. Let’s ask again: Refine network for automatic question generation. In Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 3312–3321.
  24. Unsupervised multi-hop question answering by question generation. In Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL-HLT), pages 5866–5880.
  25. Zero-shot fact verification by claim generation. In Annual Meeting of the Association for Computational Linguistics (ACL), pages 476–483.
  26. Recent advances in neural question generation.
  27. Semantic graphs for generating deep questions. In Annual Meeting of the Association for Computational Linguistics (ACL), pages 1463–1475.
  28. Bleu: a method for automatic evaluation of machine translation. In Annual Meeting of the Association for Computational Linguistics (ACL), pages 311–318. ACL.
  29. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21:140:1–140:67.
  30. Know what you don’t know: Unanswerable questions for squad. In Annual Meeting of the Association for Computational Linguistics (ACL), pages 784–789.
  31. Squad: 100, 000+ questions for machine comprehension of text. In Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2383–2392.
  32. Ashwin Ram. 1991. A theory of questions and question asking. Journal of the Learning Sciences, 1(3-4):273–318.
  33. Sudha Rao and Hal Daumé III. 2018. Learning to ask good questions: Ranking clarification questions using neural expected value of perfect information. In Annual Meeting of the Association for Computational Linguistics (ACL), pages 2737–2746.
  34. Sudha Rao and Hal Daumé III. 2019. Answer-based adversarial training for generating clarification questions. In Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL-HLT), pages 143–155.
  35. CoQA: A conversational question answering challenge. Transactions of the Association for Computational Linguistics, 7:249–266.
  36. Learning to retrieve engaging follow-up queries. In Findings of the Association for Computational Linguistics: EACL 2023, pages 2009–2016, Dubrovnik, Croatia. Association for Computational Linguistics.
  37. Summarization programs: Interpretable abstractive summarization with neural modular trees.
  38. Neural models for key phrase extraction and question generation. In Workshop on Machine Reading for Question Answering@ACL, pages 78–88.
  39. A multi-agent communication framework for question-worthy phrase extraction and question generation. In AAAI Conference on Artificial Intelligence (AAAI), pages 7168–7175.
  40. Chain of thought prompting elicits reasoning in large language models. ArXiv, abs/2201.11903.
  41. Hotpotqa: A dataset for diverse, explainable multi-hop question answering. In Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2369–2380.
  42. Paragraph-level neural question generation with maxout pointer and gated self-attention networks. In Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 3901–3910.
Citations (6)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now