From RAG to RICHES: Retrieval Interlaced with Sequence Generation (2407.00361v1)

Abstract: We present RICHES, a novel approach that interleaves retrieval with sequence generation tasks. RICHES offers an alternative to conventional RAG systems by eliminating the need for separate retriever and generator. It retrieves documents by directly decoding their contents, constrained on the corpus. Unifying retrieval with generation allows us to adapt to diverse new tasks via prompting alone. RICHES can work with any Instruction-tuned model, without additional training. It provides attributed evidence, supports multi-hop retrievals and interleaves thoughts to plan on what to retrieve next, all within a single decoding pass of the LLM. We demonstrate the strong performance of RICHES across ODQA tasks including attributed and multi-hop QA.

• The paper demonstrates that Riches unifies retrieval and generation within a single LLM, eliminating the complexities of multi-model pipelines.
• The methodology employs constrained beam search with FM-Index support, ensuring precise sequence generation based on corpus constraints.
• The system achieves notable performance in multi-hop ODQA tasks, simplifying pipelines and enhancing retrieval accuracy.

From RAG to RICHES: Retrieval Interleaved with Sequence Generation

The paper "From RAG to RICHES: Retrieval Interleaved with Sequence Generation" introduces Riches, a unified system designed to interleave retrieval tasks with sequence generation tasks. This approach fundamentally diverges from conventional Retrieval-Augmented Generation (RAG) systems, which typically require separate retrieval and generation models. Here, retrieval and generation are consolidated into a single LLM and decoding process, thereby eliminating the complexities associated with multi-system pipelines.

Introduction and Key Innovations

Riches builds on foundational work in constrained decoding for retrieval but significantly extends it to support multiple retrievals seamlessly entwined with natural language generation. Unlike traditional pipelines that interfuse diverse models, Riches unifies the tasks through an integrated approach, constrained on a corpus at each decoding step. Key observations driving this work include:

1. LLMs as Knowledge Repositories: LLMs inherently possess extensive knowledge, but they can fail to incorporate fresh information post-training and are prone to hallucination. Retrieval addresses this limitation by grounding responses in up-to-date information.
2. Decoding as Search: LLM decoding is fundamentally a search process through a vast space of potential sequences. By constraining this search to sequences known to exist in a specified corpus, retrieval can be efficiently handled within the LLM framework.
3. Unified Task Framework: Integrating retrieval with generation allows the system to quickly adapt to various new tasks through prompting alone, leveraging advances in instruction following models without necessitating additional training.

Methodology and Technical Framework

The technical approach of Riches involves constrained beam search within an autoregressive LLM. This process is highlighted by:

• Constrained Beam Decoding:

Beam search is employed for heuristic exploration within the search space, with additional logical constraints enforcing that beam continuations exist within a predefined corpus. This ensures precision in generation, particularly for constrained sequences representing retrieval keys.

• Efficient Constraints via FM-Index:

A compressed suffix array, FM-Index, supports rapid substring search operations, effectively constraining model outputs to valid corpus sequences during decoding. This contributes to scalability in scenarios involving large corpora.

• Adaptive Beam Size Strategy:

To balance constrained and unconstrained generation segments, an adaptive strategy dynamically adjusts the beam size, preserving computational resources for retrieval-centric operations while allowing for flexible natural language generation.

Retrieval Key Strategies and Evaluation

Riches' efficacy is critically dependent on the design of retrieval keys, with several strategies examined:

• Document Title and Section Headers:

Utilizing hierarchical metadata and titles as retrieval indices.

• Paragraph and Sentence Sub-strings:

Direct retrieval using substrings within paragraphs or individual sentences.

• Propositional Index:

Compact propositions serve as retrieval keys, significantly enhancing the model's alignment and efficacy in capturing and retrieving atomic information.

Evaluation encompasses both single-hop and multi-hop Open Domain Question Answering (ODQA) tasks. Metrics include answer accuracy (F1 score) and attribution (AutoAIS), with results indicating notable success in integrating retrieval within generation streams. Riches competes effectively against traditional dense retrieval methods, particularly excelling in multi-hop queries.

Implications and Future Directions

Riches marks substantial progress in the convergence of retrieval and generation tasks, promising several practical and theoretical advancements:

• Practical Impact:

The unified approach simplifies pipeline complexities, reduces latency by eliminating inter-model communication, and allows compact indexing strategies for dynamic data environments.

• Theoretical Implications:

Riches underscores the potential for LLMs to act as both knowledge stores and retrieval agents. This paves the way for advanced systems wherein LLMs autonomously navigate and manipulate extensive knowledge spaces.

Future research could focus on extending Riches to operate on corpora unseen during pre-training, exploring richer indexing strategies, and enhancing constrained decoding algorithms for improved retrieval efficiency. Potential applications span beyond ODQA to tasks involving extensive document synthesis and long-form generation, contingent on fine-tuning the retrieval process to handle more complex semantic compositions.

Riches represents a meaningful evolution in the integration of retrieval with LLM capabilities, advocating a streamlined, unified approach that holds considerable promise for next-generation natural language processing systems.