---
title: 'PyRAG: Modular RAG Frameworks'
url: https://www.emergentmind.com/topics/pyrag
type: topic
---

# PyRAG: Modular RAG Frameworks

PyRAG is an umbrella term for two distinct, state-of-the-art open-source frameworks for Retrieval-Augmented Generation (RAG): (1) the original "pyterrier_rag" declarative pipeline extension for PyTerrier focused on modular RAG construction, and (2) a more recent system treating multi-hop RAG as executable program synthesis and execution. Both systems provide formal abstractions, algorithmic toolchains, and extensibility for question answering over large text corpora, but differ in their orientation—declarative relational-algebraic pipelines versus programmatic, stepwise reasoning environments. This entry details both frameworks, articulates their underlying models, architectures, and empirical findings, and situates them within the broader evolution of retrieval-augmented generation.

## 1. The Declarative PyTerrier-RAG Framework

The original PyRAG framework is the "pyterrier_rag" plugin for the PyTerrier information retrieval platform. It integrates RAG research within a single declarative Python ecosystem, leveraging the PyTerrier operator model and relational transformers architecture. PyRAG delivers:

- A RAG-focused data model, with new relation types encoding context and answer information.
- Transformers for composing retrieved documents into prompts suitable for LLM readers (e.g., the Concatenator).
- Reader wrappers supporting HuggingFace and OpenAI backends, as well as Fusion-in-Decoder architectures.
- Built-in access to standard QA datasets (Natural Questions, TriviaQA, HotpotQA, etc.) and indices.
- Declarative, pipeline-oriented evaluation using measures such as Exact Match (EM), F₁ token overlap, ROUGE, and BERTScore.
- Extensibility for sparse (BM25), learned-sparse (SPLADE), dense (E5, ColBERT), and reranking or decoding models [2506.10802].

The PyTerrier-RAG system allows succinct construction and modification of RAG pipelines, enabling researchers to define end-to-end workflows (retrieval, reranking, prompt construction, answer generation) in a few lines of code, backed by PyTerrier’s extensible operator notation.

## 2. Relational-Algebraic Data Model and Pipeline Syntax

PyTerrier’s pipeline architecture models information flow through typed table relations:

- $Q({\tt qid}, {\tt query})$ — Set of input questions.
- $D({\tt docno}, {\tt text})$ — Document corpus representation.
- $R({\tt qid}, {\tt docno}, {\tt score}, {\tt rank})$ — Retrieval output.
- $Q_c({\tt qid}, {\tt query}, {\tt context})$ — Query with context (bundled retrievals).
- $A({\tt qid}, {\tt answer})$ — Generated answer.
- $GA({\tt qid}, [{\tt gold\_answers}])$ — Gold standard answers.

Transformers—atomic pipeline operators—map between these relations (e.g., $Q \to R$, $R \to Q_c$, $Q_c \to A$). Operator notation (“$\gg$” for sequence, “$+$” for list merge, “$\% K$” for cutoff) allows algebraic expressions of complex pipelines. For example, a typical RAG QA pipeline is expressed as:

```python
pipeline = (bm25_retriever >> monoT5_reranker) >> Concatenator() >> Reader(...)
```

This paradigm enables the combinatorial reuse of retrievers, rerankers, and readers, with seamless extension to new architectures [2506.10802].

## 3. Supported Models, Evaluation Workflows, and Empirical Findings

Supported retrieval models span:

- **Sparse:** BM25 using the Robertson–Walker formula,
  $$
  \text{score}_{BM25}(q, d)
    = \sum_{t\in q} \text{IDF}(t)
      \frac{f_{t, d} (k_1 + 1)}{f_{t, d} + k_1 (1 - b + b\,|d|/\mathit{avdl})}
  $$
- **Learned-sparse:** SPLADE.
- **Dense:** DPR-style bi-encoder, E5, ColBERT,
  $$
  \text{sim}(q, d) = \cos(\mathbf{e}_q, \mathbf{e}_d)
  $$
- **Rerankers:** monoT5, duoT5, GenRank.
- **Readers:** Fusion-in-Decoder, sequence-to-sequence models via HuggingFace, OpenAI, or custom backends.

Datasets provided via `pt.get_dataset('rag:…')` include standard open-domain QA, multi-hop (e.g., HotpotQA, 2WikiMultihopQA), dialogue, and fact-checking tasks. Evaluation is declarative; e.g.,

```python
systems = [bm25_fid, e5_fid]
topics = dataset.get_topics('dev')
gold = dataset.get_answers('dev')
measures = [pyterrier_rag.measures.F1, pyterrier_rag.measures.EM]
pt.Experiment(systems, topics, gold, measures)
```

Metrics:

- EM (“normalized” string match):
  $$
  \mathrm{EM}(a, g) = 
  \begin{cases}
    1 & \text{if normalization}(a) = \text{normalization}(g) \\
    0 & \text{otherwise}
  \end{cases}
  $$
- $F_1$ (token overlap), nDCG@k for retrieval [2506.10802].

Empirical results show, for the NQ dev set with top-10 retrieval and FiD T5-Base, an increase from BM25+T5-FiD (21.7% EM, 28.4% F₁) to E5+T5-FiD (24.8% EM, 31.9% F₁); demonstrating consistent improvements with dense retrieval.

## 4. PyRAG as Executable Multi-hop Reasoning Programs

A subsequent PyRAG system advances RAG for multi-hop QA by synthesizing explicit, executable Python programs that instantiate the reasoning process [2605.12975]. This framework proceeds as follows:

- **Decomposition Agent:** Splits the input question $q$ into atomic sub-queries (JSON list).
- **Planning Agent:** Constructs an executable program $\pi$ over two APIs:
  - $\texttt{retrieve}(query: \text{str}, topk: \text{int}) \to \text{List[str]}$
  - $\texttt{answer}(query: \text{str}, docs: \text{List[str]}) \to \text{str}$
  The final output is generated by an $\texttt{answer}$ call with all intermediate results as context.
- **Answer Agent:** For each answer call, produces an answer span, optionally citing supporting documents.

A typical multi-hop QA plan under this model consists of alternating retrieve and answer statements, with intermediate variables recorded:

```python
docs1 = retrieve("Who directed Inception?", topk=5)
director1 = answer("Who directed Inception?", docs1)
docs2 = retrieve("Who directed Jurassic Park?", topk=5)
director2 = answer("Who directed Jurassic Park?", docs2)
docs3 = retrieve(f"When was {director1} born?", topk=5)
birth1 = answer(f"When was {director1} born?", docs3)
docs4 = retrieve(f"When was {director2} born?", topk=5)
birth2 = answer(f"When was {director2} born?", docs4)
final_answer = answer(
    f"Given: {director1} was born {birth1}, "
    f"{director2} was born {birth2}. "
    "Answer the question: Who was born earlier, the director of "
    "Inception or Jurassic Park?"
)
```

Execution is modeled as a state-transformer over environments $\mathcal{E}$, incrementally binding variables and producing a fully inspectable trace for debugging and error analysis [2605.12975].

## 5. Error Correction and Adaptive Retrieval

The executable nature of this RAG formulation enables two notable forms of grounded refinement:

- **Compiler-Grounded Self-Repair:** If the program encounters a syntax or runtime error, the system surfaces the faulty code and traceback to the Planning Agent, which attempts to automatically repair the code and retry execution, iterating up to $T$ times.
- **Execution-Driven Adaptive Retrieval:** When an answer step is unsatisfactory (e.g., returns “unknown”), the pipeline automatically triggers a new retrieval with increased $k$, performing adaptive, targeted evidence gathering for subproblems.

Both mechanisms are uniquely enabled by program traceability, in contrast to free-form chain-of-thought reasoning, where intermediate states and errors are opaque [2605.12975].

## 6. Empirical Performance, Ablation Studies, and Limitations

PyRAG (program-synthesis variant) demonstrates robust empirical gains on standard QA and multi-hop benchmarks:

| Method             | PopQA | HotpotQA | 2WikiMQA | MuSiQue | Bamboogle | Avg. EM |
|--------------------|-------|----------|----------|---------|-----------|---------|
| Vanilla RAG        | 26.7  | 28.9     | 18.9     | 4.7     | 16.0      | 19.0    |
| IRCoT              | 32.6  | 32.7     | 24.8     | 9.1     | 24.3      | 24.7    |
| ITER-RETGEN        | 31.4  | 32.5     | 28.9     | 8.7     | 29.6      | 26.2    |
| **PyRAG**          | 33.5  | 34.0     | 33.4     | 11.8    | 41.5      | 30.8    |

The largest improvements occur on compositional, multi-hop QA tasks. Under RL-trained settings, PyRAG-RL matches or exceeds prior RL-based search agents (e.g., ReSearch), particularly on 2WikiMQA and Bamboogle.

Ablation analysis reveals:

- “+Execution” (actual program execution) provides the largest jump in average EM, confirming the central role of explicit, inspectable operations in multi-hop QA.
- Code-specialized backbone models (e.g., Qwen2.5, Qwen3, LLaMA-3.1) yield substantial gains only when coupled with explicit program synthesis (e.g., $+6.9$ EM on 2WikiMQA), not under vanilla RAG.
- Efficiency is improved, averaging only $3.1\!-\!3.7$ LLM calls per query (Decompose, Plan, Answer), a superior accuracy–cost tradeoff.
- Error analysis implicates retrieval-miss ($\sim$$50\%$) and intermediate propagation ($\sim$$30\%$) as the principal failure modes.

Limitations include LLM hallucination risk, retrieval recall bottlenecks, compute intensity for large indices and model calls, and the need for optimal prompt and chain design [2506.10802, 2605.12975].

## 7. Extensibility, Best Practices, and Prospective Directions

Both PyTerrier-RAG and the program-synthesis PyRAG frameworks are designed for extensibility:

- Components (retrievers, rerankers, readers) can be interchanged by variable rebinding or subclassing PyTerrier Transformers.
- New datasets are supported by providing standard dictionaries (qid, query, answer); indices can be rebuilt or reused.
- Complex reasoning strategies (multi-hop, iterative retrieval, self-refining chains) are readily supported, either via explicit pipeline algebra (PyTerrier-RAG) or programmatic chaining (executed PyRAG).
- Large-scale experiments benefit from prefix-computation caching in PyTerrier, and performance bottlenecks can be addressed with batch processing and advanced retrievers.

Future directions, as signaled by recent work, include tighter integration of iterative reasoning (IRCoT, REANO, TRACE), advanced program repair heuristics, and further harnessing code-specialized LLMs for robust, efficient multi-hop RAG [2506.10802, 2605.12975].

In summary, PyRAG—across both declarative and program-synthesis paradigms—formalizes and operationalizes the entire RAG workflow as a modular, extensible, and empirically validated research toolchain, enabling rigorous experimentation and rapid iteration in information-seeking question answering.

Source: https://www.emergentmind.com/topics/pyrag