---
title: LogicRAG Framework
url: https://www.emergentmind.com/topics/logicrag-framework
type: topic
---

# LogicRAG Framework

The LogicRAG framework is a logic-aware retrieval-augmented generation system for large language models that dispenses with pre-built graphs. Instead, it dynamically decomposes complex queries into subproblems, determines their dependencies, and orchestrates adaptive, efficient knowledge retrieval and multi-step reasoning entirely at inference time. LogicRAG is designed to improve both accuracy and resource efficiency in complex question answering settings, especially those requiring multi-hop reasoning, by modeling the query’s latent logic structure with a dynamically built directed acyclic graph (DAG) rather than a static corpus-wide graph.

## 1. Motivation and Problem Definition

Large language models are susceptible to hallucination—generating factually incorrect responses—when confronted with queries outside their training distribution or knowledge scope. Retrieval-augmented generation (RAG) mitigates this by grounding language models with relevant passages from external corpora:
\[
C = R(Q), \quad A = f_{\mathrm{LLM}}(Q \mid C)
\]
where $Q$ is a query, $K$ is a corpus, $R(\cdot)$ a retrieval function, and $f_{\mathrm{LLM}}$ the language model.

Graph-based RAG (GraphRAG) methods have leveraged offline-constructed knowledge graphs for retrieval, showing improvements on complex multi-hop questions. However, such approaches incur heavy preprocessing costs—requiring transformation of the entire corpus into a graph, consuming thousands of tokens and many minutes even for moderate corpora. Furthermore, static graphs are query-agnostic; their edges and structure may not fit the logical requirements of individual queries, leading to misaligned retrieval, inefficiency, and update latency.

LogicRAG addresses these limitations by dynamically discovering a problem-specific reasoning structure at inference time, decomposing the query, extracting a dependency DAG, and controlling retrieval generation adaptively without the need for a pre-built graph.

## 2. Dynamic Query Decomposition and DAG Construction

Central to LogicRAG is on-the-fly query decomposition. An LLM-based decomposition function
\[
f_{\mathrm{decomp}} : Q \mapsto P = \{p_1, \ldots, p_n\}
\]
segments $Q$ into minimal, non-overlapping subproblems that together cover all required knowledge. Decomposition is operationalized with few-shot prompting, e.g., “Segment the question into minimal reasoning steps.” Completeness and non-overlap are enforced.

LogicRAG then induces a directed acyclic graph
\[
G = (V, E), \quad V = \{v_i\},\; v_i \leftrightarrow p_i
\]
where edges $(v_i \rightarrow v_j) \in E$ are present if $p_j$ depends on the answer to $p_i$. Edges are inferred by prompting the LLM on stepwise dependencies, and a DFS check ensures acyclicity. This process encodes the latent reasoning pathway optimal for each query instance.

## 3. Retrieval and Reasoning Scheduling via Graph Linearization

To guide execution, LogicRAG linearizes the DAG using topological sort:
\[
\sigma = [v_{(1)}, v_{(2)}, \ldots, v_{(n)}], \quad \text{with } \mathrm{rank}(v_{(i)}) = i
\]
ensuring dependencies are resolved in order. This sequence allows the logic-respecting scheduling of retrieval and subproblem answering. The topological sort is implemented via DFS in $\mathcal{O}(|V| + |E|)$ time, as shown below:
```python
def TopoSort(G):
    visited = set()
    stack = []
    def DFS(v):
        visited.add(v)
        for u in Neighbors(v):
            if u not in visited:
                DFS(u)
        stack.append(v)
    for v in V:
        if v not in visited:
            DFS(v)
    return reversed(stack)
```
At each stage, a merged or individual subproblem is resolved, and its context retrieved in alignment with the global reasoning chain.

## 4. Pruning Mechanisms for Efficiency

LogicRAG implements two pruning strategies to optimize both accuracy and resource efficiency:

- **Graph Pruning:** At each topological level $r$, a sibling set $S^{(r)}$ of subproblems is scored for semantic similarity. If similarity $\mathrm{sim}(p_i,p_j) > \tau$, subproblems are merged into a unified query $q^{\mathrm{uni}}$ via the LLM, effectively reducing redundant retrieval.

- **Context Pruning:** As retrieval progresses, a rolling memory $\mathrm{Mem}^{(r)}$ accumulates the most salient retrieved information. It is updated by summarization:
\[
\mathrm{Mem}^{(r)} = \mathrm{Summarize}(\mathrm{Mem}^{(r-1)} \cup C^{(r)})
\]
Any retrieved passage $d$ with score $R(q,d) < \delta$ is dropped. This filtering limits token bloat and maintains a high signal-to-noise ratio for downstream reasoning.

## 5. Adaptive Retrieval and Generation Pipeline

For each (possibly merged) subproblem $q$, LogicRAG retrieves the top-$k$ passages:
\[
C = \{d_j : R(q, d_j)\}_{j=1}^k, \quad R(q, d_j)=\langle \mathrm{embed}(q),\,\mathrm{embed}(d_j)\rangle
\]
using cosine similarity between query and document embeddings.

The reasoning pipeline operates as a forward pass across sorted DAG levels. For each $r$:
- Construct $q^{(r)}$ by merging $S^{(r)}$ subproblems.
- Retrieve $C^{(r)}$.
- Summarize to update $\mathrm{Mem}^{(r)}$.
- Prompt the LLM for each $p_i \in S^{(r)}$ with the rolling memory:
  ```
  "Given rolling memory Mem^(r), answer subproblem p_i:
    [p_i]
    Context: [Mem^(r)]"
  ```
If novel subproblems are articulated by the LLM, these are dynamically added to $G$ and processed recursively, ensuring completeness.

## 6. Experimental Results, Efficiency, and Illustrative Example

LogicRAG was evaluated on HotpotQA (2-hop), 2WikiMQA (2–4 hops), and MuSiQue (composed single-hop) datasets against baselines including vanilla RAG (various $k$), zero-shot LLMs, and state-of-the-art GraphRAG-style models (KGP, RAPTOR, GraphRAG, LightRAG, HippoRAG, HippoRAG2). Key findings:
- On 2WikiMQA, string-match accuracy increased from 50.0% to 64.7% (+14.7 pp over the best baseline).
- Average token consumption on 2WikiMQA was reduced to $\sim$1.8K versus 2.8–4.7K for GraphRAG variants.
- Latency per question decreased to $\sim$9.8 seconds, compared to 13–35 seconds for graph-based methods.
- Combined pruning mechanisms (graph and context) reduced token usage per query by 60–70%, without significant impact on accuracy for $k > 5$ retrieval (with further $k$ increases offering minimal gains but linearly growing resource usage).

A three-step illustrative example is as follows. For the question, “What month did Tripartite discussions begin between Britain, France, and the country…?”:
1. Decomposition: $p_1$ (identify country), $p_2$ (decode “nobilities commonwealth”), $p_3$ (find month).
2. DAG construction: $v_2 \rightarrow v_1 \rightarrow v_3$, reflecting dependency chains.
3. Topological sort: $\sigma = [v_2, v_1, v_3]$.
4. Iterative reasoning:
   - $r=1$, $q^{(1)}$: “What historic entity does ‘nobilities commonwealth’ refer to?” (Answer: "Polish–Lithuanian Commonwealth")
   - $r=2$, $q^{(2)}$: “Given Mem$^{(1)}$, what country did Warsaw Pact leadership originate from?” (Answer: "the Soviet Union")
   - $r=3$, $q^{(3)}$: “When (month) did Tripartite discussions (Britain, France, Soviet Union) begin?” (Answer: "June")
   - Context pruning distills $\sim$600 tokens of retrieved content per round to $\sim$150 core tokens, while graph pruning was not needed in this instance.

## 7. Significance and Context within Retrieval-Augmented Reasoning

LogicRAG demonstrates that adaptive, inference-stage logic modeling outperforms prior static GraphRAG approaches both in answer quality and efficiency. By eschewing any global, pre-computed graph, it minimizes preprocessing overhead, reduces per-query resource consumption, and enables dynamic alignment between retrieval structure and query logic. The methodology also underscores the greater generality and extensibility of logic-structured retrieval—facilitating multi-step, dependency-aware reasoning for arbitrary queries encountered at inference time. Extensive benchmarks show that dynamic logic-aware retrieval both outperforms prior pre-built graph baselines and achieves 30–60% savings in token and latency costs for complex multi-hop question answering tasks [2508.06105].

Source: https://www.emergentmind.com/topics/logicrag-framework