Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Reranking Passages with Coarse-to-Fine Neural Retriever Enhanced by List-Context Information (2308.12022v2)

Published 23 Aug 2023 in cs.CL

Abstract: Passage reranking is a critical task in various applications, particularly when dealing with large volumes of documents. Existing neural architectures have limitations in retrieving the most relevant passage for a given question because the semantics of the segmented passages are often incomplete, and they typically match the question to each passage individually, rarely considering contextual information from other passages that could provide comparative and reference information. This paper presents a list-context attention mechanism to augment the passage representation by incorporating the list-context information from other candidates. The proposed coarse-to-fine (C2F) neural retriever addresses the out-of-memory limitation of the passage attention mechanism by dividing the list-context modeling process into two sub-processes with a cache policy learning algorithm, enabling the efficient encoding of context information from a large number of candidate answers. This method can be generally used to encode context information from any number of candidate answers in one pass. Different from most multi-stage information retrieval architectures, this model integrates the coarse and fine rankers into the joint optimization process, allowing for feedback between the two layers to update the model simultaneously. Experiments demonstrate the effectiveness of the proposed approach.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (35)
  1. 2017. An attention mechanism for neural answer selection using a combined global and local view. In Proceedings of ICTAI 2017. IEEE.
  2. 2017. Efficient cost-aware cascade ranking in multi-stage retrieval. In Proceedings of ACM SIGIR.
  3. 2018. Convolutional neural networks for soft-matching n-grams in ad-hoc search. In Proceedings of WSDM. ACM.
  4. 2019. BERT: pre-training of deep bidirectional transformers for language understanding. In Proceedings of NAACL-HLT, 4171–4186. Association for Computational Linguistics.
  5. 2016. A deep relevance matching model for ad-hoc retrieval. In Proceedings of the 25th ACM International on Conference on Information and Knowledge Management. ACM.
  6. 2019. Decoupled weight decay regularization. In Proceedings of ICLR.
  7. 2018. Query driven algorithm selection in early stage retrieval. In Proceedings of WSDM. ACM.
  8. 2016. Key-value memory networks for directly reading documents. In Proceedings of EMNLP, 1400–1409. The Association for Computational Linguistics.
  9. 2017. Learning to match using local and distributed representations of text for web search. In Proceedings of WWW.
  10. 2022. Text and code embeddings by contrastive pre-training. arXiv preprint arXiv:2201.10005.
  11. 2016. MS MARCO: A human generated machine reading comprehension dataset. In Proceedings of the Workshop on Cognitive Computation: Integrating neural and symbolic approaches 2016 co-located with the 30th Annual Conference on Neural Information Processing Systems (NIPS 2016), Barcelona, Spain, December 9, 2016, volume 1773 of CEUR Workshop Proceedings. CEUR-WS.org.
  12. 2019. Passage re-ranking with bert. arXiv preprint arXiv:1901.04085.
  13. 2016. A decomposable attention model for natural language inference. In Proceedings of EMNLP, 2249–2255. The Association for Computational Linguistics.
  14. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of EMNLP-IJCNLP, 3980–3990. Association for Computational Linguistics.
  15. 2016. Reasoning about entailment with neural attention. In Bengio, Y., and LeCun, Y., eds., Proceedings of ICLR.
  16. 2018. Multi-mention learning for reading comprehension with neural cascades. In Proceedings of ICLR. OpenReview.net.
  17. 2018. Context-aware answer sentence selection with hierarchical gated recurrent neural networks. IEEE/ACM Transactions on Audio, Speech, and Language Processing 26(3):540–549.
  18. 2016. Convolutional neural networks vs. convolution kernels: Feature engineering for answer sentence reranking. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 1268–1278.
  19. 2017. Attention is all you need. In Advances in Neural Information Processing Systems.
  20. 2017. A compare-aggregate model for matching text sequences. In Proceedings of ICLR. OpenReview.net.
  21. 2021. Kepler: A unified model for knowledge embedding and pre-trained language representation. Transactions of the Association for Computational Linguistics 9:176–194.
  22. 2017. Bilateral multi-perspective matching for natural language sentences. In Proceedings of IJCAI, 4144–4150. ijcai.org.
  23. 2016. Inner attention based recurrent neural networks for answer selection. In Proceedings of ACL 2016.
  24. 2023. C-pack: Packaged resources to advance general chinese embedding. arXiv preprint arXiv:2309.07597.
  25. 2017. End-to-end neural ad-hoc ranking with kernel pooling. In Proceedings of the 40th International ACM SIGIR conference on research and development in information retrieval, 55–64. ACM.
  26. 2016. Hierarchical attention networks for document classification. In Proceedings of NAACL 2016.
  27. 2015. Wikiqa: A challenge dataset for open-domain question answering. In Màrquez, L.; Callison-Burch, C.; Su, J.; Pighin, D.; and Marton, Y., eds., Proceedings of EMNLP, 2013–2018. The Association for Computational Linguistics.
  28. 2016. Abcnn: Attention-based convolutional neural network for modeling sentence pairs. Transactions of the Association for Computational Linguistics 4:259–272.
  29. 2021. A collaborative ai-enabled pretrained language model for aiot domain question answering. IEEE Transactions on Industrial Informatics 18(5):3387–3396.
  30. 2022. Switchnet: A modular neural network for adaptive relation extraction. Computers and Electrical Engineering 104:108445.
  31. Zhu, H. 2022a. Financial data analysis application via multi-strategy text processing. arXiv preprint arXiv:2204.11394.
  32. Zhu, H. 2022b. Metaaid: A flexible framework for developing metaverse applications via ai technology and human editing. arXiv preprint arXiv:2204.01614.
  33. Zhu, H. 2023a. Fqp 2.0: Industry trend analysis via hierarchical financial data. arXiv preprint arXiv:2303.02707.
  34. Zhu, H. 2023b. Metaaid 2.0: An extensible framework for developing metaverse applications via human-controllable pre-trained models. arXiv preprint arXiv:2302.13173.
  35. Zhu, H. 2023c. Metaaid 2.5: A secure framework for developing metaverse applications via large language models. arXiv preprint arXiv:2312.14480.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (1)
  1. Hongyin Zhu (16 papers)

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now