KiRAG: Knowledge-Driven Iterative Retriever for Enhancing Retrieval-Augmented Generation (2502.18397v1)

Abstract: Iterative retrieval-augmented generation (iRAG) models offer an effective approach for multi-hop question answering (QA). However, their retrieval process faces two key challenges: (1) it can be disrupted by irrelevant documents or factually inaccurate chain-of-thoughts; (2) their retrievers are not designed to dynamically adapt to the evolving information needs in multi-step reasoning, making it difficult to identify and retrieve the missing information required at each iterative step. Therefore, we propose KiRAG, which uses a knowledge-driven iterative retriever model to enhance the retrieval process of iRAG. Specifically, KiRAG decomposes documents into knowledge triples and performs iterative retrieval with these triples to enable a factually reliable retrieval process. Moreover, KiRAG integrates reasoning into the retrieval process to dynamically identify and retrieve knowledge that bridges information gaps, effectively adapting to the evolving information needs. Empirical results show that KiRAG significantly outperforms existing iRAG models, with an average improvement of 9.40% in R@3 and 5.14% in F1 on multi-hop QA.

Enhancing Retrieval-Augmented Generation with Knowledge-Driven Iterative Retrieval

The paper “Knowledge-Driven Iterative Retriever for Enhancing Retrieval-Augmented Generation” presents \OURS{}, a model aimed at improving retrieval-augmented generation (RAG) systems' performance in multi-hop question answering (QA). This research addresses key challenges in iterative retrieval processes by leveraging a knowledge-driven methodology centered around knowledge triples. The challenges identified involve disruptions in retrieval due to irrelevant documents and the static nature of retrievers which fail to adapt dynamically to continually evolving information needs.

Model Architecture and Methodology

\OURS{} enhances iterative RAG models by incorporating a knowledge-driven iterative retrieval framework that utilizes knowledge triples, which are extracted by decomposing documents into triples of the form $\langle$head entity, relation, tail entity$\rangle$. This approach aims to reduce inaccuracies and noise that typically affect the retrieval process, thereby ensuring a more reliable integration of knowledge into the multi-hop QA tasks. The proposed model iteratively builds a reasoning chain informed by these knowledge triples, which aids the retrieval of interconnected pieces of information.

Key components of \OURS{} include:

1. Knowledge Decomposition: Documents are pre-processed to extract knowledge triples. This allows the formation of a knowledge graph corpus that supports efficient retrieval.
2. Iterative Retrieval Framework: The iterative retrieval approach relies on a combination of reasoning chain aligners and constructors. The aligner identifies relevant triples based on the evolving context, while the constructor uses these triples to build reasoning chains iteratively.
3. Document Ranking: The knowledge triples are then used to rank documents according to their relevance, which optimizes the retrieval process for the QA tasks.

Empirical Results

Empirical results demonstrate that \OURS{} achieves significant improvements in both retrieval and QA performance when compared to existing RAG models. Across tested datasets, \OURS{} shows an average improvement of 9.40% in R$\text{@}$3 and 5.14% in F1 scores for multi-hop QA tasks. These results suggest that the model effectively addresses the issues of noise and static retrieval present in traditional systems.

Implications and Future Research Directions

The practical implications of this research are substantial, offering a more reliable method for integrating and retrieving relevant information in RAG tasks, which is crucial in real-world applications such as automated customer support and information retrieval systems. Theoretical implications include expanding the understanding of how knowledge triples can aid in dynamically adapting retrieval needs in LLMs.

Future work could explore further tuning of the reasoning chain aligner and constructor by leveraging additional supervised data or even unsupervised learning methods. Furthermore, there exists a fruitful avenue in expanding this methodology to other domains beyond multi-hop QA, such as dialog systems and complex decision-making frameworks that rely upon contextual understanding and knowledge integration.

By mitigating common retrieval challenges, \OURS{} provides a robust framework that can potentially enhance future developments in AI systems requiring sophisticated reasoning capabilities.