The Power of Noise: Redefining Retrieval for RAG Systems (2401.14887v4)

Abstract: Retrieval-Augmented Generation (RAG) has recently emerged as a method to extend beyond the pre-trained knowledge of LLMs by augmenting the original prompt with relevant passages or documents retrieved by an Information Retrieval (IR) system. RAG has become increasingly important for Generative AI solutions, especially in enterprise settings or in any domain in which knowledge is constantly refreshed and cannot be memorized in the LLM. We argue here that the retrieval component of RAG systems, be it dense or sparse, deserves increased attention from the research community, and accordingly, we conduct the first comprehensive and systematic examination of the retrieval strategy of RAG systems. We focus, in particular, on the type of passages IR systems within a RAG solution should retrieve. Our analysis considers multiple factors, such as the relevance of the passages included in the prompt context, their position, and their number. One counter-intuitive finding of this work is that the retriever's highest-scoring documents that are not directly relevant to the query (e.g., do not contain the answer) negatively impact the effectiveness of the LLM. Even more surprising, we discovered that adding random documents in the prompt improves the LLM accuracy by up to 35%. These results highlight the need to investigate the appropriate strategies when integrating retrieval with LLMs, thereby laying the groundwork for future research in this area.

Introduction

Advancements in LLMs have brought about remarkable capabilities in text generation and understanding, yet their constraint in managing large contexts heralds a limitation. RAG systems aim to surmount this challenge by enabling access to external, dynamically sourced information during response generation. In comprehensive analysis, the paper examines the integral role of the IR phase in a RAG setup, posing an essential research question on optimizing a retriever for effective RAG prompts, focusing on retriever document types—relevant, related and irrelevant—to the prompts.

Retrieval-Augmented Generation Systems

The RAG system composition enhances factual information generation by supplementing the LLM power with an IR component. A key advantage is the increase of the effective context size for LLMs. This dynamic retrieval enriches the input for the generative module, impacting the response's accuracy. The core inquiry is the role of the retriever, exploring its ideal characteristics for prompt optimization. The paper breaks new ground by not just considering the relevance but also the position of retrieved documents, and the surprising benefits of including irrelevant documents.

Experimental Insights

The paper meticulously assesses the IR phase's impact, revealing that related documents harm RAG system performance more than unrelated ones. The counterintuitive discovery is that irrelevant documents, when included in the context, can improve accuracy by up to 35%. Different configurations of the proximity of the gold document to the query are explored, and it is observed that nearby placement enhances LLM performance. These findings challenge the conventional perceivable utility of retrieved documents and advocate for reconsidering information retrieval strategies for RAG system optimization.

Future Directions

These insights demand a systematic rethinking of IR strategies within RAG frameworks. Given that LLMs can manage a finite number of documents, retrievers should supply a minimal set of documents, balancing relevant contents with a certain allowance for irrelevant material, which surprisingly tends to increase accuracy. Moreover, the paper calls for research on the apparent effectiveness of random, irrelevant documents in enhancing the efficiency of LLM responses within RAG systems. The work encourages future research to explore why noise in the system can be beneficial and to delineate the nuanced characteristics that contribute to this unexpected utility.