Multi-Granularity Guided Fusion-in-Decoder (2404.02581v1)
Abstract: In Open-domain Question Answering (ODQA), it is essential to discern relevant contexts as evidence and avoid spurious ones among retrieved results. The model architecture that uses concatenated multiple contexts in the decoding phase, i.e., Fusion-in-Decoder, demonstrates promising performance but generates incorrect outputs from seemingly plausible contexts. To address this problem, we propose the Multi-Granularity guided Fusion-in-Decoder (MGFiD), discerning evidence across multiple levels of granularity. Based on multi-task learning, MGFiD harmonizes passage re-ranking with sentence classification. It aggregates evident sentences into an anchor vector that instructs the decoder. Additionally, it improves decoding efficiency by reusing the results of passage re-ranking for passage pruning. Through our experiments, MGFiD outperforms existing models on the Natural Questions (NQ) and TriviaQA (TQA) datasets, highlighting the benefits of its multi-granularity solution.
- Evidentiality-guided generation for knowledge-intensive NLP tasks. In NAACL, pages 2226–2243.
- Andrew P. Bradley. 1997. The use of the area under the ROC curve in the evaluation of machine learning algorithms. Pattern Recognit., 30:1145–1159.
- Reading wikipedia to answer open-domain questions. In ACL, pages 1870–1879.
- Fido: Fusion-in-decoder optimized for stronger performance and faster inference. In Findings of the ACL, pages 11534–11547.
- Retrieval augmented language model pre-training. In ICML, pages 3929–3938.
- Fid-light: Efficient and effective retrieval-augmented text generation. In SIGIR, pages 1437–1447.
- Gautier Izacard and Edouard Grave. 2021a. Distilling knowledge from reader to retriever for question answering. In ICLR.
- Gautier Izacard and Edouard Grave. 2021b. Leveraging passage retrieval with generative models for open domain question answering. In EACL, pages 874–880.
- Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension. In ACL, pages 1601–1611.
- Grape: Knowledge graph enhanced passage reader for open-domain question answering. In Findings of EMNLP, pages 169–181.
- Dense passage retrieval for open-domain question answering. In EMNLP, pages 6769–6781.
- Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In ICLR.
- Natural questions: a benchmark for question answering research. Trans. Assoc. Comput. Linguistics, pages 452–466.
- Fid-ex: Improving sequence-to-sequence models for extractive rationale generation. In EMNLP, pages 3712–3727.
- You only need one model for open-domain question answering. In EMNLP, pages 3047–3060.
- Latent retrieval for weakly supervised open domain question answering. In ACL, pages 6086–6096.
- Retrieval-augmented generation for knowledge-intensive NLP tasks. In NeurIPS.
- Focal loss for dense object detection. In ICCV, pages 2999–3007.
- Grancats: Cross-lingual enhancement through granularity-specific contrastive adapters. In CIKM, pages 1461–1471.
- Rodrigo Nogueira and Kyunghyun Cho. 2019. Passage re-ranking with bert. CoRR.
- Rocketqa: An optimized training approach to dense passage retrieval for open-domain question answering. In NAACL-HLT, pages 5835–5847.
- Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res., 21:140:1–140:67.
- Is chatgpt good at search? investigating large language models as re-ranking agents. In EMNLP, pages 14918–14937.
- Wiebke Wagner. 2010. Steven bird, ewan klein and edward loper: Natural language processing with python, analyzing text with the natural language toolkit - o’reilly media, beijing, 2009, ISBN 978-0-596-51649-9. Lang. Resour. Evaluation, 44:421–424.
- Rfid: Towards rational fusion-in-decoder for open-domain question answering. In Findings of ACL, pages 2473–2481.
- Kg-fid: Infusing knowledge graph in fusion-in-decoder for open-domain question answering. In ACL, pages 4961–4974.
- Eunseong Choi (8 papers)
- Hyeri Lee (4 papers)
- Jongwuk Lee (24 papers)