When RAG Meets Query Planning: Logical Query Trees for Resolving Exploratory Reasoning Problems

Abstract: Retrieval-Augmented Generation (RAG) effectively grounds LLMs in external knowledge but struggles with \textbf{exploratory reasoning problems (ERPs)} that are the complex queries involving high uncertainty and ambiguity. Resolving ERPs requires complex reasoning with unclear paths, tending to result in retrieval noise and error accumulation. Furthermore, the absence of an end-to-end planning mechanism makes it difficult to generate effective trajectories for ERPs. Motivated by database query planning, we introduce \emph{PlanRAG}, an RAG framework that models ERPs of natural language as \textbf{logical query trees (LQTs)}. However, translating ERPs into LQTs is non-trivial due to representation and optimization gaps between structured SQL and unstructured natural language, making it highly challenging to construct high-quality LQTs. To address these problems, we first decompose ERPs into atomic queries and then organize them into LQTs using dynamic programming guided by a cost model involving multiple complementary dimensions. Finally, we execute iterative aggregation, rewriting, retrieval, and generation over LQTs, processing nodes concurrently and propagating intermediate results upward, with further parallelization across multiple threads for efficiency. Our experimental results show that PlanRAG outperforms state-of-the-art iteration-based and graph-based RAG systems on our newly constructed dataset, \textbf{WikiWeb-ERP}, thereby providing a new formulation for optimizing natural language queries. Our source code and dataset are available at https://anonymous.4open.science/r/PlanRAG-main-B2C8/.

• The paper introduces PlanRAG, a framework that resolves exploratory reasoning problems by decomposing queries into atomic units structured as logical query trees.
• It employs dynamic programming and LLM-driven decomposition to optimize query plans, achieving 3–5 point gains in accuracy on complex ERP benchmarks.
• The system features parallelized execution and relation preprocessing, reducing LLM calls and token costs while enhancing interpretability and robustness.

PlanRAG: Logical Query Trees for Optimizing Exploratory Reasoning in RAG

Introduction

This paper addresses critical limitations in Retrieval-Augmented Generation (RAG) when tackling exploratory reasoning problems (ERPs). Current RAG pipelines, though effective for conventional multi-hop and knowledge-grounded tasks, exhibit poor performance in scenarios with high uncertainty, open-ended reasoning structures, and implicit task decompositions. Such ERPs, characterized by ambiguous and intertwined sub-queries, induce excessive retrieval noise, error propagation, and lack a principled planning mechanism.

Drawing a direct analogy with database query optimization, the authors instantiate PlanRAG—a framework that formalizes ERPs as Logical Query Trees (LQTs). This approach enables explicit modeling, global planning, and structured execution over decomposed atomic queries, bridging the representational and optimization gaps between structured query planning and unstructured natural language ERPs.

Methodology

Atomic Query Decomposition

PlanRAG leverages LLMs for initial query analysis, decomposing the input ERP into minimal atomic queries, each corresponding to a single (subject, relation, object) triple. This atomicity ensures precise mapping of information needs and supports direct evidence retrieval, while preserving complex dependencies. Atomic queries are classified as parent–child, sibling, or unrelated, informing the dependency structure in the planning stage.

LQT Construction via Dynamic Programming

Translating the principles of relational query planning to natural language, PlanRAG invokes a variant of Selinger’s dynamic programming (DP) algorithm to construct the LQT:

• Relation preprocessing statically filters out unrelated atomic query pairs, effectively pruning the search space and minimizing redundant LLM calls.
• Cycle prevention ensures tree acyclicity during node merges, maintaining dag-structured logical plans.
• Context-aware merging uses LLMs in prompt loops, providing partial tree context to local merge assessments, thus enforcing global semantic coherence.

Structural and semantic optimality of candidate LQTs is evaluated with a novel multi-dimensional cost model, defined over:

• Tree size (favoring maximal query coverage),
• Structural density (richness of dependencies),
• Tree depth and balance (favoring parallelism and preventing accumulation bottlenecks),
• Semantic similarity (alignment with original ERP intent, measured with dense embeddings—BGE).

The final plan $\mathcal{T}^*$ is the minimal-cost LQT under these metrics, balancing global structure and semantic fidelity.

Parallelized Execution and Pipeline Integration

Physical execution proceeds bottom-up over the LQT: leaf atomic queries are retrieved and answered, while internal nodes aggregate child results, rewrite contextualized queries, and repeat retrieval/generation. Importantly, PlanRAG schedules independent LQT nodes in parallel, exploiting the inherent tree structure for efficient inference and reduced latency.

Prompts and LLM orchestrations (for decomposition, relation classification, merging decisions, rewriting, and tree-to-text transformations) are tightly sequenced, enabling explicit, modular, and debuggable pipeline orchestration.

Empirical Analysis

Dataset: WikiWeb-ERP

To substantiate PlanRAG’s benefits, the authors introduce WikiWeb-ERP, a comprehensive benchmark purposely designed for ERPs. Sourced from challenging questions in established multi-hop QA datasets (BrowseComp, GAIA, etc.) and synthetically expanded with WebSailor protocols, WikiWeb-ERP offers over 3500 queries spanning both curated and live web documents, providing a highly suitable testbed for complex exploratory reasoning in open-domain RAG.

Results and Ablations

PlanRAG achieves robust improvements over state-of-the-art iterative (RetGen, DualRAG, GenGround, KiRAG) and graph-based (ChainRAG, HopRAG, LEGO-GraphRAG) RAG paradigms. On WikiWeb-ERP, PlanRAG surpasses the strongest baselines by 3-5 points in accuracy, F1, EM, and semantic correctness, with the gains magnified under increased query complexity.

Ablation studies confirm that:

• Disabling relation preprocessing affects efficiency, but has limited impact on accuracy;
• Cycle prevention and context-aware merging are critical for LQT robustness;
• Semantic similarity is the dominant factor in cost modeling, as its removal leads to the steepest performance degradation.

PlanRAG’s architecture is verified across LLaMA (2/3, 7B–70B) and Qwen (2.5, 7B–72B) families, as well as with popular retrievers (BM25, BGE, Contriever, ColBERT), demonstrating parameter- and retriever-agnostic efficacy.

Efficiency experiments show that relation preprocessing reduces LLM calls by $>$5$\times$ and token costs by $>$2.5$\times$, while parallel execution further cuts wall-clock times by %%%%5$\times$6%%%% relative to sequential baselines.

Further Analyses

• PlanRAG maintains quality under simulated noise at each pipeline stage, with accuracy degrading gracefully rather than catastrophically.
• LQT quality, judged by both human annotation and GPT-4o assessment, exceeds direct LLM plan generation due to explicit structural constraints and semantic anchoring.
• System resource consumption (runtime, GPU memory) scales predictably with ERP complexity.
• Theoretical analysis confirms that pairwise relation pruning yields quadratic, rather than exponential, LLM invocation scaling.

Error Attribution

The main failure modes are in (i) ambiguous dependency recognition in atomic queries and relation classification (planning errors), (ii) non-optimal or spurious evidence selection (retrieval errors), and (iii) partial inconsistencies in global synthesis of atomic answers.

Theoretical and Practical Implications

Theoretical Implications

The formalization of ERPs as LQTs aligns natural language reasoning with well-established database principles, offering:

• A principled mapping from unstructured query intents to explicit, interpretable logical plans.
• Global optimization via cost-driven search, with trade-offs between coverage, robustness, and parallelizability.
• The capacity to prevent spurious spatiotemporal or semantic entanglements that arise in purely chain-based or undirected-graph methods.

PlanRAG demonstrates that strong reasoning gains can be realized without reliance on manual trajectory annotation or policy learning, supporting training-free, deterministic, and audit-friendly system design.

Practical Implications

PlanRAG is suited for real-world applications where queries are open-ended, evidence is abundant and noisy, and response accuracy/groundedness is paramount (e.g., research agents, scientific QA, legal/medical reasoning). Its architectural features promote:

• Robust parallelism and resource-efficient planning, crucial for deployment at scale or in latency-sensitive contexts.
• Modular integration with diverse LLMs and retrievers.
• Enhanced interpretability through explicit plan visualization and diagnostics.

Future Directions

The work points to several potential extensions:

• Reducing residual LLM dependency by incorporating lighter-weight or fine-tuned planning modules.
• Integrating physical optimization analogs (e.g., intermediate result caching) for further efficiency gains.
• Adapting the cost model for domain- or application-specific contextualization.
• Extending the formalism to physical plans capable of system-level optimizations and full-stack deployment.

Conclusion

PlanRAG presents a rigorous, database-inspired formulation for end-to-end reasoning over exploratory queries in RAG. By constructing and optimizing LQTs, it achieves substantial gains in both accuracy and efficiency over leading RAG architectures on explicitly challenging ERP datasets. Its core theoretical contribution—the unification of database query planning with unstructured natural language reasoning—offers a blueprint for future research on planning-aware, controllable, and interpretable LLM-augmented systems.

Reference:

"When RAG Meets Query Planning: Logical Query Trees for Resolving Exploratory Reasoning Problems" (2607.00508)