Investigating the Factual Knowledge Boundary of Large Language Models with Retrieval Augmentation (2307.11019v3)

Abstract: LLMs have shown impressive prowess in solving a wide range of tasks with world knowledge. However, it remains unclear how well LLMs are able to perceive their factual knowledge boundaries, particularly under retrieval augmentation settings. In this study, we present the first analysis on the factual knowledge boundaries of LLMs and how retrieval augmentation affects LLMs on open-domain question answering (QA), with a bunch of important findings. Specifically, we focus on three research questions and analyze them by examining QA, priori judgement and posteriori judgement capabilities of LLMs. We show evidence that LLMs possess unwavering confidence in their knowledge and cannot handle the conflict between internal and external knowledge well. Furthermore, retrieval augmentation proves to be an effective approach in enhancing LLMs' awareness of knowledge boundaries. We further conduct thorough experiments to examine how different factors affect LLMs and propose a simple method to dynamically utilize supporting documents with our judgement strategy. Additionally, we find that the relevance between the supporting documents and the questions significantly impacts LLMs' QA and judgemental capabilities. The code to reproduce this work is available at https://github.com/RUCAIBox/LLM-Knowledge-Boundary.

Investigating the Factual Knowledge Boundary of LLMs with Retrieval Augmentation

This paper focuses on dissecting the capabilities of LLMs in understanding their factual knowledge boundaries, particularly in the context of open-domain question answering (QA), when augmented with retrieval mechanisms. The work scrutinizes the self-awareness of LLMs like ChatGPT concerning their knowledge limits, focusing on both the models' innate capacities and their interaction with external retrieval-augmented inputs.

Core Findings and Contributions

The paper lays out three primary research questions addressing the extent of LLMs' awareness regarding their knowledge limits, the impacts of retrieval augmentation, and the influence of document characteristics on LLM performance. Here are the distilled insights from the investigations:

1. Knowledge Boundary Perception: LLMs tend to overestimate their ability to accurately answer questions, often proceeding with certainty even when unsure or incorrect. This indicates a significant gap between confidence and actual knowledge, underscoring the need for improved self-evaluation mechanisms in these models.
2. Impact of Retrieval Augmentation: Introducing retrieval-augmented frameworks significantly enhances LLMs' performance in QA tasks, accentuating their awareness of knowledge boundaries. By leveraging dense and sparse retrieval methods or even documents produced by other LLMs, the models exhibit improved judgment, offering a promising strategy to harness more accurate and contextually relevant content.
3. Quality of Supporting Documents: LLMs show a marked tendency to rely on provided documents when crafting responses. This reliance is heavily contingent upon the quality and relevance of these documents. High-quality supporting documents improve both the models' performance and their self-evaluation accuracy. Conversely, irrelevant documents can mislead the models, degrading output quality.

Experimental Framework and Evaluation

The paper employs multiple datasets, including Natural Questions, TriviaQA, and HotpotQA, utilizing both sparse and dense retrieval methods alongside LLM-generated content. Evaluation metrics encompass traditional QA performance indicators like exact match and F1 scores but importantly incorporate metrics for self-evaluation accuracy and judgmental quality. By examining diverse retrieval settings and document qualities, the research delineates the nuanced interplay between internal model knowledge and external augmentation.

Theoretical and Practical Implications

Theoretical Implications: This work offers insights into the intrinsic limitations of LLMs in self-assessing their knowledge, which has profound implications for future model development. Enhancing self-awareness functionalities will be crucial for developing more autonomous and reliable AI systems. It also adds to the ongoing discussion regarding the interpretability and accountability of AI systems, raising questions about how these models can better recognize and communicate their limitations.

Practical Implications: On the practical front, the enhanced performance observed with retrieval augmentation suggests that LLM deployment strategies should incorporate dynamic retrieval mechanisms. Particularly, dynamically adjusting the extent of reliance on retrieval, depending on the model's confidence, could optimize performance without unnecessarily increasing computational costs.

Future Directions

Based on the findings, future advancements could focus on refining retrieval techniques to better discern document quality and relevance, and integrating adaptive mechanisms that allow LLMs to modulate their confidence threshold dynamically. Moreover, exploring hybrid models that combine LLMs with more precise retrieval modules might strike an optimal balance between independent reasoning and supplemental inference from external corpora.

In conclusion, this research underscores the critical role of retrieval augmentation in expanding the effective knowledge boundaries of LLMs. It paves the way for more informed interactions between model-encoded knowledge and external data, ultimately enhancing model performance and trustworthiness in knowledge-intensive applications.