Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Pregnant Questions: The Importance of Pragmatic Awareness in Maternal Health Question Answering (2311.09542v2)

Published 16 Nov 2023 in cs.CL

Abstract: Questions posed by information-seeking users often contain implicit false or potentially harmful assumptions. In a high-risk domain such as maternal and infant health, a question-answering system must recognize these pragmatic constraints and go beyond simply answering user questions, examining them in context to respond helpfully. To achieve this, we study assumptions and implications, or pragmatic inferences, made when mothers ask questions about pregnancy and infant care by collecting a dataset of 2,727 inferences from 500 questions across three diverse sources. We study how health experts naturally address these inferences when writing answers, and illustrate that informing existing QA pipelines with pragmatic inferences produces responses that are more complete, mitigating the propagation of harmful beliefs.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (41)
  1. Dorit Abusch. 2002. Lexical alternatives as a source of pragmatic presuppositions. In Semantics and linguistic theory, volume 12, pages 1–19.
  2. Nicholas Allott. 2018. Conversational implicature.
  3. Akari Asai and Eunsol Choi. 2021. Challenges in information-seeking QA: Unanswerable questions and paragraph retrieval. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 1492–1504, Online. Association for Computational Linguistics.
  4. Task-aware retrieval with instructions. arXiv preprint arXiv:2211.09260.
  5. Adrien Barbaresi. 2021. Trafilatura: A Web Scraping Library and Command-Line Tool for Text Discovery and Extraction. In Proceedings of the Joint Conference of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing: System Demonstrations, pages 122–131. Association for Computational Linguistics.
  6. David Ian Beaver. 1997. Presupposition. In Handbook of logic and language, pages 939–1008. Elsevier.
  7. Gay Becker and Edwina Newsom. 2003. Socioeconomic status and dissatisfaction with health care among chronically ill african americans. American journal of public health, 93(5):742–748.
  8. Scaling instruction-finetuned language models. arXiv preprint arXiv:2210.11416.
  9. QAFactEval: Improved QA-based factual consistency evaluation for summarization. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2587–2601, Seattle, United States. Association for Computational Linguistics.
  10. Herbert P Grice. 1975. Logic and conversation. In Speech acts, pages 41–58. Brill.
  11. Are natural language inference models IMPPRESsive? Learning IMPlicature and PRESupposition. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8690–8705, Online. Association for Computational Linguistics.
  12. Mistral 7b.
  13. Jad Kabbara and Jackie Chi Kit Cheung. 2022. Investigating the performance of transformer-based NLI models on presuppositional inferences. In Proceedings of the 29th International Conference on Computational Linguistics, pages 779–785, Gyeongju, Republic of Korea. International Committee on Computational Linguistics.
  14. Dense passage retrieval for open-domain question answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6769–6781, Online. Association for Computational Linguistics.
  15. (qa)2: Question answering with questionable assumptions. ArXiv, abs/2212.10003.
  16. Which linguist invented the lightbulb? presupposition verification for question-answering. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 3932–3945, Online. Association for Computational Linguistics.
  17. Todor Koev. 2018. Notions of at-issueness. Language and Linguistics Compass, 12(12):e12306.
  18. Natural questions: A benchmark for question answering research. Transactions of the Association for Computational Linguistics, 7:452–466.
  19. Pragmatics. Cambridge university press.
  20. Chin-Yew Lin. 2004. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out, pages 74–81, Barcelona, Spain. Association for Computational Linguistics.
  21. What makes good in-context examples for GPT-3? In Proceedings of Deep Learning Inside Out (DeeLIO 2022): The 3rd Workshop on Knowledge Extraction and Integration for Deep Learning Architectures, pages 100–114, Dublin, Ireland and Online. Association for Computational Linguistics.
  22. Practical guidance for the development of rosie, a health education question-and-answer chatbot for new mothers. Journal of Public Health Management and Practice, 29(5):663–670.
  23. Cross-task generalization via natural language crowdsourcing instructions. In ACL.
  24. Training language models to follow instructions with human feedback.
  25. Physicians’ perceptions of chatbots in health care: cross-sectional web-based survey. Journal of medical Internet research, 21(4):e12887.
  26. NOPE: A corpus of naturally-occurring presuppositions in English. In Proceedings of the 25th Conference on Computational Natural Language Learning, pages 349–366, Online. Association for Computational Linguistics.
  27. Stanley Peters. 2016. Speaker commitments: presupposition. In Semantics and Linguistic Theory, pages 1083–1098.
  28. Christopher Potts. 2004. The logic of conventional implicatures, volume 7. OUP Oxford.
  29. Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics.
  30. Harnessing the linguistic signal to predict scalar inferences. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5387–5403, Online. Association for Computational Linguistics.
  31. BLEURT: Learning robust metrics for text generation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7881–7892, Online. Association for Computational Linguistics.
  32. Towards understanding sycophancy in language models. arXiv preprint arXiv:2310.13548.
  33. Ben Shneiderman. 1992. Tree visualization with tree-maps: 2-d space-filling approach. ACM Transactions on graphics (TOG), 11(1):92–99.
  34. Prompting gpt-3 to be reliable.
  35. Mpnet: Masked and permuted pre-training for language understanding. CoRR, abs/2004.09297.
  36. Pragmatic presuppositions. In Proceedings of the Texas conference on per~ formatives, presuppositions, and implicatures. Arlington, VA: Center for Applied Linguistics, pages 135–148. ERIC.
  37. Robert S Taylor. 1962. The process of asking questions. American documentation, 13(4):391–396.
  38. Unsupervised dense information retrieval with contrastive learning. In Transactions on Machine Learning Research.
  39. A critical evaluation of evaluations for long-form question answering. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3225–3245, Toronto, Canada. Association for Computational Linguistics.
  40. Crepe: Open-domain question answering with false presuppositions. ArXiv, abs/2211.17257.
  41. GRICE: A grammar-based dataset for recovering implicature and conversational rEasoning. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 2074–2085, Online. Association for Computational Linguistics.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (7)
  1. Neha Srikanth (6 papers)
  2. Rupak Sarkar (11 papers)
  3. Heran Mane (1 paper)
  4. Elizabeth M. Aparicio (1 paper)
  5. Quynh C. Nguyen (2 papers)
  6. Rachel Rudinger (46 papers)
  7. Jordan Boyd-Graber (68 papers)

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now