---
title: 'Plan*RAG: Structured Planning in RAG Pipelines'
url: https://www.emergentmind.com/topics/plan-rag
type: topic
---

# Plan*RAG: Structured Planning in RAG Pipelines

Plan*RAG is a collective term for a family of frameworks and methodologies that explicitly introduce symbolic or structured planning to Retrieval-Augmented Generation (RAG) pipelines. Plan*RAG architectures decompose complex tasks into sequenced, often atomic, sub-problems before invoking retrieval, aiming to systematically mitigate error propagation and reasoning drift in knowledge-intensive tasks including multi-hop question answering, code generation, decision-making, and domain-specific planning. This approach is realized in a variety of domains (e.g., text QA, embodied AI, software engineering, medical planning), and shares a characteristic “plan-then-retrieve” or “plan-augmented-retrieve” pattern, contrasting with earlier “retrieve-then-generate” RAG. Recent Plan*RAG instantiations demonstrate improved accuracy, interpretability, and robustness across zero-shot and domain-adapted settings.

## 1. Core Principles and Motivations

Plan*RAG introduces planning as an explicit, structured intermediary between the user’s query and the retrieval-augmented generation steps. Whereas classic RAG pipelines condition generation directly on retrieved knowledge chunks, Plan*RAG approaches first produce a formalized plan—such as a reasoning DAG, sequential subgoals, or domain-specific pseudocode—that then guides or parameterizes retrieval:

- **Query Decomposition**: The input is decomposed into atomic, typically single-hop, sub-queries or reasoning steps that collectively define the high-level reasoning path required to answer the original query [2410.20753], [2502.18139], [2504.16787], [2406.14979].
- **Planning Outside LM Context**: The reasoning plan is generated and maintained external to the LLM’s context window, circumventing context-length limitations associated with in-context chain-of-thought prompting. For example, “Plan*RAG” formalizes the plan as a Directed Acyclic Graph (DAG) whose sub-nodes correspond to atomic queries or facts [2410.20753].
- **Error Localization and Attribution**: By explicitly separating planning, retrieval, and aggregation, Plan*RAG architectures enable targeted verification and multi-granularity consistency checks at each reasoning step [2504.16787].
- **Adaptivity and Efficiency**: Modular design allows for plug-and-play integration with a range of retrievers, generators, and evaluators, and supports parallel execution of plan steps [2410.20753].

This paradigm addresses major RAG limitations: fragmented reasoning chains, context overflow, error compounding across multi-hop tasks, and an inability to attribute final outputs to discrete evidentiary supports.

## 2. Representative Plan*RAG Architectures

A diversity of Plan*RAG systems have been developed, sharing a “plan–retrieve–generate” backbone with domain-specific extensions:

| System           | Planning Mechanism              | Retrieval Strategy             | Domain/Application     |
|------------------|--------------------------------|-------------------------------|-----------------------|
| Plan*RAG [2410.20753]   | Test-time DAG generation, atomic subqueries | Per-node retrieval, Relevance/Critic experts | Multi-hop QA |
| PAR RAG [2504.16787]    | Top-down plan decomposition, JSON trace    | Multi-granularity (coarse+fine) per sub-question | Multi-hop QA |
| LevelRAG [2502.18139]   | Symbolic high-level searcher, iterative logic planning | Hybrid (sparse/dense/web), query rewriting | QA (single/multi-hop) |
| RPG [2406.14979]        | Iterative plan-token prediction, plan–answer cycles | Fine-grained selection, multi-task prompt tuning | Knowledge-intensive QA |
| PlanRAG [2406.12430]    | Explicit subgoal generation for decision analysis | Plan-driven SQL/Cypher queries | Decision-making QA   |
| PERC [2412.12447]       | Pseudocode plan-based retrieval, plan as query | Semantic retrieval over plan representations | Code generation     |
| ThreatLens [2505.06821] | Multi-agent LLM planners for threat/policy/test plan | Vector RAG, iterative user–agent loop | Hardware security   |
| MedPlan [2503.17900]    | Strict SOAP-inspired (Assessment→Plan) planning | Plan- and history-level retrieval | Medical plan generation |
| P-RAG [2409.11279]      | Progressive, iterative plan–retrieve cycles | Scene+goal similarity, growing DB | Embodied task planning |
| Plan+RAG-Code [2408.08335] | DSL plan structuring, function- and few-shot retrieval | API/function metadata + example code | NL to DSL/Automation |

A common pattern is sequential or iterative execution: a plan is composed (by the LLM or auxiliary planner), each plan element triggers a tailored retrieval and generation step (often re-ranked or filtered for relevance), results are aggregated, and—when required—verifiers or critics introduce plan revision or error correction.

## 3. Formal Models and Algorithmic Structures

Most Plan*RAG systems model planning as a symbolic or partially symbolic process:

- **DAG-Based Reasoning (Plan*RAG)** [2410.20753]: The plan is a DAG $\mathcal{G} = (V, E)$ with nodes $q_i$ corresponding to atomic queries. For each $q$, the system instantiates the sub-query $\tilde{q}$ by injecting the answers of parent nodes, retrieves supporting documents, and generates $G(q)$, the sub-answer. Parallelization is enabled for nodes at the same depth.
- **State-Transition Model (LevelRAG)** [2502.18139]: High-level planning states $s_t$ track both the set of resolved subqueries and their interim summaries; actions include decomposition (“decompose”), summarization, verification, and supplementation. Search is terminated when all subqueries’ summaries are judged sufficient for final answer synthesis.
- **Plan-then-Act-and-Review (PAR RAG)** [2504.16787]: The plan is a structured sequence $P = \{(\mathrm{thought}_i, q_i)\}$; at each step coarse- and fine-grained retrievals are performed, followed by consistency checks. The Action module executes sub-queries; the Review module iteratively verifies or revises sub-answers via multi-passage citation overlap.
- **Iterative Plan–Answer Cycles (RPG)** [2406.14979]: A plan token $p_t$ specifies the next subtopic, guiding paragraph-level retrieval for an answer segment $y_t$. Plan–answer iteration continues until output completion or early stopping.
- **Explicit Plan for Data-Driven QA (PlanRAG)** [2406.12430]: LLM emits a stepwise plan $P$; each step is translated into SQL/Cypher database queries for observation, then LLM integrates observations using business rules; re-planning is invoked as needed.
- **Plan-as-Query Retrieval (PERC)** [2412.12447]: Code examples are mapped to pseudocode plans, and retrieval is performed over plan representations; retrieved examples are converted as needed to the target programming language.

## 4. Domain-Specific Adaptations and Applications

Plan*RAG methodology has been adapted for a range of knowledge-intensive settings:

- **Multi-hop and Long-form Question Answering**: Direct evidence for improved multi-step reasoning fidelity, error localization, and overall accuracy. For example, Plan*RAG [2410.20753] improves HotpotQA accuracy from 25.49% (standard RAG) to 35.67% and F1 from 31.22 to 39.68, while PAR RAG [2504.16787] achieves relative EM/F1 uplifts of +31.6% and +37.9% over state-of-the-art baselines on HotpotQA and MuSiQue.
- **Decision-Making over Structured Data**: PlanRAG outperforms prior iterative RAG by +15.8 pp in Locating and +7.4 pp in Building scenarios on the Decision QA benchmark [2406.12430].
- **Medical Plan Generation**: MedPlan’s “Plan × RAG” mirrors clinician workflow by first producing an assessment, then using retrieved cross-patient plus self-history SOAP records to generate personalized treatment plans, yielding up to +0.3183 BLEU and +0.5213 METEOR (Medical-Mixtral-7B-v2k) [2503.17900].
- **Threat Modeling and Hardware Verification**: ThreatLens employs multi-agent planners (threat, policy, plan generation) with RAG, reducing manual effort ~75% and achieving 92% precision in threat filtering on NEORV32 SoC [2505.06821].
- **Code and DSL Generation**: PERC’s plan-as-query retrieval outperforms code retrieval baselines in both in- and cross-language settings, e.g., in MultiPL-E, Ruby: 67.27% (RepoCoder) → 69.81%, Lua: 60.81% → 64.10% [2412.12447]; Plan+RAG for DSL generation matches fine-tuned baselines in-domain and exceeds them by +7 pts similarity on out-of-domain APIs [2408.08335].
- **Embodied AI**: P-RAG’s iterative, database-augmented planning improves unseen task success rates on ALFRED: GPT-4 (no retrieval) 7.05% → P-RAG after 3 iters 14.11%, and with self-iteration up to 27.4% [2409.11279].

## 5. Theoretical and Empirical Impact

Plan*RAG approaches advance RAG systems by:

- **Reducing Error Propagation**: Top-down planning, coupled with per-step verification, prevents local retrieval/generation failures from corrupting downstream reasoning.
- **Improving Attribution**: Atomic subqueries, each linked to a discrete retrieved document, provide strong evidence traceability—76% of answers are exact substrings of the retrieved doc in Plan*RAG [2410.20753].
- **Enhancing Modular Integration**: Plug-and-play design supports deployment atop arbitrary LLMs, retrievers (BM25, DPR, Contriever), and verification modules, requiring minimal or no model fine-tuning [2410.20753], [2502.18139].
- **Enabling Scalability and Efficiency**: Parallel plan step execution and context-bounded node retrieval mitigate context window overflow and reduce unnecessary retrievals, as with the Critic Expert in Plan*RAG (retrieval calls reduced by 19% with negligible accuracy loss) [2410.20753].
- **Performance Gains**: Across domains, Plan*RAG variants match or exceed proprietary models (e.g., LevelRAG surpasses GPT4o and ReSP) [2502.18139], show significant performance uplift compared to vanilla one-pass RAG, and exhibit enhanced generalization to low-resource regimes.

## 6. Limitations, Challenges, and Future Directions

Notable limitations identified in Plan*RAG research include:

- **Computational Overhead**: Multi-step planning, per-step retrieval, review modules, and verification add latency (e.g., PAR RAG average RTPQ ≈ 26s) and increase inference cost [2504.16787].
- **Planning Quality Sensitivity**: Poor initial plan decomposition or specification can cause retrieval to miss critical evidence or narrow the search space excessively [2406.14979], [2412.12447].
- **Database Scalability and Memory**: Progressive accumulation of trajectories or intermediate plans may cause database growth and potential retrieval efficiency degradation, as seen in P-RAG [2409.11279].
- **Limits of Current LLM Reasoners**: P-RAG and similar systems plateau as LLM reasoning capabilities (particularly for embodied, non-textual tasks) saturate [2409.11279].
- **Automatic Plan Extraction**: The quality of LLM-generated plans or pseudocode may be a failure point (e.g., erroneous plan steps, unreliable pseudocode extraction in PERC) [2412.12447].

Ongoing work investigates learned retriever/reranker modules, adaptive granularity planning, plan critics or quality validators, efficient memory condensation, and cross-modal plan representations (e.g., integrating vision directly in embodied settings). A plausible implication is that tighter coupling between learned planning agents and retrieval subsystems, or joint end-to-end optimization as in trainable consistency/verifier modules, could further enhance accuracy and robustness.

## 7. Summary Table: Plan*RAG Systems and Key Features

| System           | Planning            | Retrieval         | Result/Claim                             | Reference        |
|------------------|--------------------|-------------------|------------------------------------------|------------------|
| Plan*RAG         | Test-time DAG       | Per-node, atomic  | +2–6 Acc/F1 on multi-hop QA              | [2410.20753]     |
| LevelRAG         | Symbolic searcher   | Hybrid (S/W/D)    | Outperforms GPT4o, F1 up to 69.33%       | [2502.18139]     |
| PAR RAG          | JSON plan, review   | Multi-granular    | +31.6% EM over baseline on HotpotQA      | [2504.16787]     |
| MedPlan          | SOAP plan           | Patient+history   | BLEU up to 0.3183, 66% ↑ clinical eval   | [2503.17900]     |
| ThreatLens       | Multi-agent plan    | Vector, iterative | 75% manual effort ↓, 92% precision       | [2505.06821]     |
| PERC             | Pseudocode plan     | Plan-as-query     | +1–5 pp Pass@1 on underrepresented PLs   | [2412.12447]     |
| PlanRAG          | Stepwise plan       | SQL/Cypher gen    | +15.8pp / +7.4pp accuracy on DQA         | [2406.12430]     |
| RPG              | Plan token per step | Paragraph select  | +8.5 F1 (2Wiki), +9.1 ROUGE (ASQA)       | [2406.14979]     |
| P-RAG            | Progressive planning| History+goal/scene| +7% SR (ALFRED: 7.05%→14.11%→27.4%)     | [2409.11279]     |
| Plan+RAG-Code    | DSL plan, function  | Example+API meta  | +7 pts sim on OOD API DSL generation     | [2408.08335]     |

## References

- Plan*RAG: Efficient Test-Time Planning for Retrieval Augmented Generation [2410.20753]
- LevelRAG: Enhancing Retrieval-Augmented Generation with Multi-hop Logic Planning over Rewriting Augmented Searchers [2502.18139]
- Credible plan-driven RAG method for Multi-hop Question Answering [2504.16787]
- MedPlan: A Two-Stage RAG-Based System for Personalized Medical Plan Generation [2503.17900]
- ThreatLens: LLM-guided Threat Modeling and Test Plan Generation for Hardware Security Verification [2505.06821]
- PERC: Plan-As-Query Example Retrieval for Underrepresented Code Generation [2412.12447]
- PlanRAG: A Plan-then-Retrieval Augmented Generation for Generative Large Language Models as Decision Makers [2406.12430]
- Retrieve-Plan-Generation: An Iterative Planning and Answering Framework for Knowledge-Intensive LLM Generation [2406.14979]
- Progressive Retrieval Augmented Generation For Planning on Embodied Everyday Task [2409.11279]
- Plan with Code: Comparing approaches for robust NL to DSL generation [2408.08335]

Source: https://www.emergentmind.com/topics/plan-rag