Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
102 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Evidentiality-aware Retrieval for Overcoming Abstractiveness in Open-Domain Question Answering (2304.03031v6)

Published 6 Apr 2023 in cs.AI

Abstract: The long-standing goal of dense retrievers in abtractive open-domain question answering (ODQA) tasks is to learn to capture evidence passages among relevant passages for any given query, such that the reader produce factually correct outputs from evidence passages. One of the key challenge is the insufficient amount of training data with the supervision of the answerability of the passages. Recent studies rely on iterative pipelines to annotate answerability using signals from the reader, but their high computational costs hamper practical applications. In this paper, we instead focus on a data-centric approach and propose Evidentiality-Aware Dense Passage Retrieval (EADPR), which leverages synthetic distractor samples to learn to discriminate evidence passages from distractors. We conduct extensive experiments to validate the effectiveness of our proposed method on multiple abstractive ODQA tasks.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (48)
  1. Learning to retrieve reasoning paths over wikipedia graph for question answering. In Proceedings of ICLR.
  2. Ms marco: A human generated machine reading comprehension dataset. arXiV preprint.
  3. Petr Baudiš and Jan Šedivý. 2015. Modeling of the question answering task in the yodaqa system. In Experimental IR Meets Multilinguality, Multimodality, and Interaction.
  4. Natural language processing with Python: analyzing text with the natural language toolkit. O’Reilly Media, Inc.
  5. Reading wikipedia to answer open-domain questions. In Proceedings of ACL.
  6. Danqi Chen and Wen-tau Yih. 2020. Open-domain question answering. In Proceedings of ACL.
  7. Counterfactual samples synthesizing for robust visual question answering. In Proceedings of CVPR.
  8. C2l: Causally contrastive learning for robust text classification. In Proceedings of AAAI.
  9. Less is more: Attention supervision with counterfactuals for text classification. In Proceedings of EMNLP.
  10. ELECTRA: Pre-training text encoders as discriminators rather than generators. In Proceedings of ICLR.
  11. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of NAACL.
  12. Glam: Efficient scaling of language models with mixture-of-experts. In Proceedings of ICML.
  13. Causal inference in natural language processing: Estimation, prediction, interpretation and beyond. Transactions of the Association for Computational Linguistics.
  14. Tevatron: An efficient and flexible toolkit for dense retrieval. arXiV preprint.
  15. Generative language models and automated influence operations: Emerging threats and potential mitigations. arXiV preprint.
  16. Retrieval augmented language model pre-training. In Proceedings of ICML.
  17. Distilling the knowledge in a neural network. In NIPS Deep Learning and Representation Learning Workshop.
  18. Poly-encoders: Transformer architectures and pre-training strategies for fast and accurate multi-sentence scoring. In Proceedings of ICLR.
  19. Gautier Izacard and Edouard Grave. 2021a. Distilling knowledge from reader to retriever for question answering. In Proceedings of ICLR.
  20. Gautier Izacard and Edouard Grave. 2021b. Leveraging passage retrieval with generative models for open domain question answering. In Proceedings of EACL.
  21. Few-shot Learning with Retrieval Augmented Language Models. arXiv preprint.
  22. Learning representations for counterfactual inference. In Proceedings of ICML.
  23. Jeff Johnson and Hervé Douze, Matthijs andJégou. 2019. Billion-scale similarity search with GPUs. IEEE Transactions on Big Data, 7(3):535–547.
  24. TriviaQA: A large scale distantly supervised challenge dataset for reading comprehension. In Proceedings of ACL.
  25. Dense passage retrieval for open-domain question answering. In Proceedings of EMNLP.
  26. Unifiedqa: Crossing format boundaries with a single qa system. In Findings of EMNLP.
  27. Relevance-guided supervision for openqa with colbert. Transactions of the Association for Computational Linguistics.
  28. Natural questions: A benchmark for question answering research. Transactions of the Association for Computational Linguistics, pages 452–466.
  29. Latent retrieval for weakly supervised open domain question answering. In Proceedings of ACL.
  30. Robustifying multi-hop QA through pseudo-evidentiality training. In Proceedings of ACL.
  31. Large language models with controllable working memory. In Findings of ACL.
  32. Efficient training of retrieval models using negative cache. Advances in Neural Information Processing Systems, 34:4134–4146.
  33. Tie-Yan Liu. 2009. Learning to rank for information retrieval. Foundations and Trends in Information Retrieval, 3(3):225–331.
  34. On the risk of misinformation pollution with large language models.
  35. Learning robust dense retrieval models from incomplete relevance labels. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval.
  36. Rocketqa: An optimized training approach to dense passage retrieval for open-domain question answering. In Proceedings of NAACL.
  37. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research.
  38. Some simple effective approximations to the 2-poisson model for probabilistic weighted retrieval. In Proceedings of the 17th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’94.
  39. End-to-end training of neural retrievers for open-domain question answering. In Proceedings of ACL-IJCNLP.
  40. End-to-end training of multi-document reader and retriever for open-domain question answering. In Advances in Neural Information Processing Systems.
  41. ColBERTv2: Effective and efficient retrieval via lightweight late interaction. In Proceedings of NAACL.
  42. Replug: Retrieval-augmented black-box language models. arXiV preprint.
  43. Nikita Spirin and Jiawei Han. 2012. Survey on web spam detection: Principles and algorithms. SIGKDD Explor. Newsl.
  44. CORE: Cooperative training of retriever-reranker for effective dialogue response selection. In Proceedgins of ACL.
  45. Unsupervised feature learning via non-parametric instance-level discrimination. In Proceedings of CVPR.
  46. Approximate nearest neighbor negative contrastive learning for dense text retrieval. In Proceedings of ICLR.
  47. Answering complex open-domain questions with multi-hop dense retrieval. In Proceedings of ICLR.
  48. HotpotQA: A dataset for diverse, explainable multi-hop question answering. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (7)
  1. Yongho Song (5 papers)
  2. Dahyun Lee (1 paper)
  3. Myungha Jang (5 papers)
  4. Seung-won Hwang (59 papers)
  5. Kyungjae Lee (37 papers)
  6. Dongha Lee (63 papers)
  7. Jinyeong Yeo (2 papers)