Do You Know What You Are Talking About? Characterizing Query-Knowledge Relevance For Reliable Retrieval Augmented Generation (2410.08320v1)

Abstract: LLMs (LMs) are known to suffer from hallucinations and misinformation. Retrieval augmented generation (RAG) that retrieves verifiable information from an external knowledge corpus to complement the parametric knowledge in LMs provides a tangible solution to these problems. However, the generation quality of RAG is highly dependent on the relevance between a user's query and the retrieved documents. Inaccurate responses may be generated when the query is outside of the scope of knowledge represented in the external knowledge corpus or if the information in the corpus is out-of-date. In this work, we establish a statistical framework that assesses how well a query can be answered by an RAG system by capturing the relevance of knowledge. We introduce an online testing procedure that employs goodness-of-fit (GoF) tests to inspect the relevance of each user query to detect out-of-knowledge queries with low knowledge relevance. Additionally, we develop an offline testing framework that examines a collection of user queries, aiming to detect significant shifts in the query distribution which indicates the knowledge corpus is no longer sufficiently capable of supporting the interests of the users. We demonstrate the capabilities of these strategies through a systematic evaluation on eight question-answering (QA) datasets, the results of which indicate that the new testing framework is an efficient solution to enhance the reliability of existing RAG systems.

• The paper introduces an online testing procedure using statistical tests like MSS, k-NN, and entropy metrics to detect out-of-knowledge queries in real time.
• The paper presents an offline two-sample Goodness-of-Fit test that identifies shifts in query distribution, flagging potential corpus misalignment.
• Empirical results across eight datasets show that while domain-specific models excel in retrieval, they struggle to reliably detect query relevance.

Query-Knowledge Relevance in Retrieval Augmented Generation

The paper "Do You Know What You Are Talking About? Characterizing Query-Knowledge Relevance For Reliable Retrieval Augmented Generation" presents a statistical framework for assessing the relevance between user queries and the knowledge corpora in Retrieval Augmented Generation (RAG) systems. The paper addresses a core issue in RAG: the dependency of generation quality on query-document relevance and the challenges it poses in safety-critical domains, such as healthcare.

Core Contributions

The authors propose a two-fold testing framework to determine how well a query can be addressed within a RAG system:

1. Online Testing Procedure:
   • An online mechanism is designed to identify out-of-knowledge queries using hypothesis testing. This procedure employs a variety of test statistics, including Maximum Similarity Score (MSS), k-Nearest Neighbor (KNN), and entropy-based metrics, to evaluate the statistical relevance of a query against a set of in-knowledge queries.
   • By introducing synthetic in-knowledge queries, the framework can operate without requiring a pre-established empirical distribution, allowing for real-time detection of low-relevance queries.
2. Offline Testing Procedure:
   • An offline mechanism assesses shifts in query distribution using a two-sample Goodness-of-Fit (GoF) test. This helps identify when the knowledge corpus may no longer align with user queries, indicating potential corpus obsolescence.

Empirical Evaluation

The authors conducted experiments using eight QA datasets across both general and biomedical domains. Key findings include:

• The proposed statistical tests reliably captured query relevance, outperforming both Local Outlier Factor (LOF) and relevance scores generated by LMs such as GPT-3.5 and GPT-4.
• Synthetic in-knowledge queries effectively approximated the true in-knowledge distribution, offering a practical solution for deploying the online testing framework.
• There was a notable misalignment between an embedding model's ability to retrieve relevant documents and its ability to detect out-of-knowledge queries. Domain-specific models, while effective in retrieval, exhibited limitations in distinguishing query relevance.

Implications and Future Directions

This work has both theoretical and practical implications. Theoretically, it lays a statistical foundation for query relevance assessment in RAG systems. Practically, it underscores the importance of query knowledge alignment, especially in contexts where misinformation could have severe consequences.

Future research could focus on refining the alignment between in-knowledge query distributions and user needs through adaptive corpora updates. Broader exploration of embedding models suited for both retrieval and relevance detection, especially in domain-specific applications, would enhance the robustness of RAG systems.

In conclusion, this paper contributes significantly to the optimization of RAG systems, providing a statistical approach to improve the reliability and safety of AI deployments across various domains.