Generative Multi-hop Retrieval (2204.13596v3)

Abstract: A common practice for text retrieval is to use an encoder to map the documents and the query to a common vector space and perform a nearest neighbor search (NNS); multi-hop retrieval also often adopts the same paradigm, usually with a modification of iteratively reformulating the query vector so that it can retrieve different documents at each hop. However, such a bi-encoder approach has limitations in multi-hop settings; (1) the reformulated query gets longer as the number of hops increases, which further tightens the embedding bottleneck of the query vector, and (2) it is prone to error propagation. In this paper, we focus on alleviating these limitations in multi-hop settings by formulating the problem in a fully generative way. We propose an encoder-decoder model that performs multi-hop retrieval by simply generating the entire text sequences of the retrieval targets, which means the query and the documents interact in the LLM's parametric space rather than L2 or inner product space as in the bi-encoder approach. Our approach, Generative Multi-hop Retrieval(GMR), consistently achieves comparable or higher performance than bi-encoder models in five datasets while demonstrating superior GPU memory and storage footprint.

• The paper introduces a generative model for multi-hop retrieval to bypass the embedding bottleneck and error propagation inherent in bi-encoder approaches.
• It utilizes an encoder-decoder architecture that generates textual sequences directly, leading to superior or comparable performance across five diverse datasets.
• Corpus memorization techniques, including LM and multi-hop memorization, further enhance the model's recall and efficiency in realistic retrieval scenarios.

Generative Multi-hop Retrieval: A Paradigm Shift in Text Retrieval

Introduction

The effectiveness of text retrieval systems is crucial for a variety of applications, ranging from search engines to question answering systems. While the bi-encoder architecture has been a dominant approach for both no-hop and multi-hop retrieval tasks, it encounters significant limitations as the complexity of the task increases, particularly for multi-hop retrieval. Recognizing these challenges, this paper presents an innovative approach titled Generative Multi-hop Retrieval (GMR), which redefines how retrieval tasks, especially those requiring multiple hops, are approached.

Limitations of Bi-Encoder Approaches

Bi-encoder architectures suffer from two major limitations in multi-hop retrieval settings. First, there's an embedding bottleneck issue: as the query becomes longer with each hop, compressing it into a fixed-size vector becomes increasingly challenging, leading to a loss of information. Secondly, bi-encoders are especially prone to error propagation; errors in early retrieval hops can severely impair the retrieval quality in subsequent hops. These challenges necessitate a reconsideration of the retrieval architecture for multi-hop scenarios.

Generative Multi-hop Retrieval (GMR)

The Generative Multi-hop Retrieval approach proposes to solve these issues by utilizing an encoder-decoder model that generates textual sequences of the retrieval targets directly, rather than encoding them into a fixed-size vector space. This approach allows for richer interactions between queries and documents, leveraging the entire parametric space of the model. GMR exhibits several advantages:

• It consistently achieves comparable or superior performance across five datasets when contrasted with bi-encoder models.
• GMR shows strength in settings that resemble real-world scenarios or those with a low rate of unseen queries during testing, underlining its practical applicability.
• Notably, the model demonstrates remarkable efficiency in GPU memory usage and storage requirements, offering significant improvements over traditional bi-encoder approaches.

Corpus Memorization Techniques

To enhance the effectiveness of GMR, the paper introduces two corpus memorization methods: LM memorization and multi-hop memorization. These techniques aim to improve the model's recall of the target corpus, thereby enhancing retrieval performance. LM memorization serves as an intermediate task, encouraging the model to memorize texts in the corpus using a standard LLMing objective. Multi-hop memorization, on the other hand, is a task that maximizes the probability of retrieving a sequence of texts given a pseudo-multi-hop query. These methods significantly contribute to the robustness and adaptability of GMR in multi-hop retrieval tasks.

Evaluation and Results

The paper presents a thorough evaluation of GMR against traditional bi-encoder models across various datasets and retrieval settings. The results highlight GMR's superior performance, particularly in dynamic retrieval scenarios where the number of hops is not predetermined. Such findings reinforce the argument that generative approaches, as exemplified by GMR, offer a viable and effective alternative to bi-encoder methods for complex retrieval tasks.

Implications and Future Directions

The success of GMR suggests several important directions for future research. One area involves exploring optimization strategies to further enhance the efficiency and performance of generative retrieval models. Another promising avenue is the development of advanced techniques that can fully exploit the generative retrieval paradigm, potentially addressing the remaining gaps identified in datasets like HotpotQA where bi-encoder models still hold an edge.

Conclusion

Generative Multi-hop Retrieval marks a significant departure from traditional bi-encoder approaches to text retrieval, addressing their fundamental limitations in the multi-hop context. By facilitating direct text sequence generation for retrieval targets, GMR not only achieves high performance across diverse datasets but also offers greater efficiency and reduced error propagation. As the field of text retrieval continues to evolve, generative approaches like GMR beckon as the next frontier, promising more adaptable and powerful retrieval systems for the future.