LevelRAG: Multi-Level RAG Framework

• LevelRAG is a family of retrieval-augmented generation architectures that incorporates multi-level reasoning, retrieval abstraction, and query planning to address traditional RAG limitations.
• It enables multi-hop logic planning, adaptive retrieval across various abstraction levels, and SLA-driven resource allocation to improve system robustness and efficiency.
• Empirical evaluations demonstrate significant gains in F1 score, answer correctness, and operational cost compared to conventional RAG systems on diverse QA benchmarks.

LevelRAG is a family of Retrieval-Augmented Generation (RAG) architectures and evaluation methodologies that introduce explicit control or awareness of multi-level reasoning, retrieval abstraction, query planning, or resource allocation. The defining motif across LevelRAG research is the incorporation of multiple abstraction levels—either in system structure, logic planning, or evaluation—which enables more effective, robust, or interpretable retrieval-augmented generation for question answering (QA) and related tasks.

1. Background and Motivation

Traditionally, RAG systems combine a retriever (e.g., dense vector, sparse BM25, or hybrid search) and a generator (e.g., LLM) to answer queries using both model parameters and externally retrieved text. Despite empirical successes, classic RAG pipelines face key limitations: query rewriting is tightly coupled to the retriever type; context windows are restricted, especially in multi-hop or hybrid settings; and there is limited awareness of question difficulty or system-level constraints.

LevelRAG methodologies emerge to address these deficits by (i) decomposing complex questions with multi-hop logic, (ii) supporting multi-level chunk abstractions in retrieval, (iii) adaptively allocating effort based on query difficulty or Service Level Agreements (SLAs), and (iv) harmonizing retrieval across vertical and horizontal system scaling (Zheng et al., 28 Jan 2025, Zhang et al., 25 Feb 2025, Iannelli et al., 7 Dec 2024, Carmel et al., 18 Nov 2025).

2. Multi-Hop Logic Planning and Decoupled Searchers

A core instantiation of LevelRAG, as presented in "LevelRAG: Enhancing Retrieval-Augmented Generation with Multi-hop Logic Planning over Rewriting Augmented Searchers," is a system architecture that separates high-level logic planning from low-level retriever-specific execution (Zhang et al., 25 Feb 2025).

• The High-Level Searcher (LLM-based) decomposes the user query $q$ into a sequence of atomic sub-questions $\{\bar{q}_1,..., \bar{q}_m\}$, independently of the downstream retrieval modality.
• Each atomic query is handled in parallel by a Sparse Searcher (BM25 with Lucene-enhanced rewrite), a Dense Searcher (embedding-based HyDE/ITRG methods with iterative pseudo-document generation), and a Web Searcher (API-driven web snippet retrieval).
• Summarization and verification steps aggregate and validate the intermediate answers, iteratively supplementing retrieval until the original query is covered.
• This architecture enables hybrid retrieval workflows, avoids retriever-interaction entanglement, and supports end-to-end multi-hop reasoning.

Experimental results show that LevelRAG outperforms prior RAG baselines and even proprietary models (e.g., GPT-4o) on single- and multi-hop QA datasets, with substantial F1 gains and improved coverage for both dense and sparse retrieval tasks (Zhang et al., 25 Feb 2025).

3. Retrieval Over Multiple Abstraction Levels

The Multiple Abstraction Level Retrieval-Augmented Generation (MAL-RAG) framework, frequently referred to as LevelRAG in literature, extends RAG pipelines by representing the corpus at four distinct abstraction levels: multi-sentence, paragraph, section, and document (Zheng et al., 28 Jan 2025).

• The Indexing Stage segments the corpus, producing chunk indices at each level using segmentation and map-reduce LLM summarization.
• At inference, each abstraction level is searched independently using a shared embedding model. Chunks are cosine-scored, softmax-normalized, and greedily accumulated across levels under global length and probability mass thresholds.
• Retrieved contexts from all levels are merged (coarse to fine granularity), preserving both high-level overviews and detailed facts, and submitted to the LLM for answer generation.

MAL-RAG demonstrates empirical improvement in correctness F1 of 25.74 percentage points over vanilla RAG baselines in domain-specific QA (e.g., Glycoscience), with each layer contributing complementary evidence and recall. Ablations confirm that merging multiple abstraction levels outperforms any single chunking granularity (Zheng et al., 28 Jan 2025).

4. Difficulty-Aware and Adaptive LevelRAG Evaluation

The LiveRAG benchmark provides the first large-scale QA dataset with explicit per-item difficulty and discriminability metrics, enabling LevelRAG systems that dynamically adjust retrieval and generation based on question complexity (Carmel et al., 18 Nov 2025).

• Each question is labeled with an IRT-derived difficulty parameter $b_i$ and discriminability $a_i$ via two-parameter logistic modeling:

$$P_{ij}(\theta_j) = \frac{1}{1 + \exp[-a_i(\theta_j - b_i)]}$$

• LevelRAG systems can leverage $b_i$ to modulate resource allocation:
  • Shallow retrieval ($k=5$) and lower confidence thresholds for easy items ($b_i \geq 0.5$).
  • Deeper retrieval ($k=25-50$) and stricter acceptance for hard items ($b_i < -1.0$).
  • Adaptive generation styles, answer lengths, and even curriculum-learning schedules for robust multi-level generalization.

This approach ensures both system efficiency (minimizing retrieval/generation overhead on trivial questions) and fidelity on challenging queries, as measured by stratified correctness scores and discriminability-weighted system ranking (Carmel et al., 18 Nov 2025).

5. SLA-Aware, Reconfigurable Multi-Agent LevelRAG

LevelRAG is also framed as a dynamic, SLA-driven orchestration layer atop traditional RAG pipelines (Iannelli et al., 7 Dec 2024). Resource allocation, retrieval thresholds, agent ensemble size, and arbitration strategies are determined at query time by explicit Service Level Objectives (SLOs) $(L_{\max}, Q_{\min}, C_{\max})$:

• The planning module solves an optimization problem:
  • Minimize cost $C_{\text{sys}}(x)$,
  • Subject to latency $L_{\text{sys}}(x) \leq L_{\max}$ and quality $Q_{\text{sys}}(x) \geq Q_{\min}$,
  • Where $x$ encodes number of agents, retrieval methods, arbitration thresholds, and other configuration parameters.
• The architecture supports:
  • Vertical scaling: specialized preprocessors and retrieval strategies per agent.
  • Horizontal scaling: variable ensemble size for high-confidence aggregation.
  • Intent awareness: query classification for differentiated pipeline control.

Empirical analyses demonstrate that LevelRAG can adapt agent count, selection thresholds, and retrieval strategies to meet diverse operational SLOs, balancing F1, hallucination rates, cost, and latency. The framework generalizes to real-world constraints, favoring high parallelism for low-latency tasks and more conservative arbitration for domains requiring high factual consistency (Iannelli et al., 7 Dec 2024).

6. Empirical Evaluations and Comparative Performance

LevelRAG algorithms have been empirically validated across a spectrum of open-domain QA and domain-specific benchmarks:

System/Approach	Dataset	Key Metric(s)	LevelRAG Result	Baseline Comparisons
Multi-hop logic planning	PopQA, HotpotQA	F1, Acc	F1 65.52–78.21, Acc 66.98–76.12	Outperforms GPT4o, RankRAG, Self-RAG (Zhang et al., 25 Feb 2025)
Multiple abstraction levels	Glycoscience QA	Answer Correctness F1	68.79% (Δ+25.74)	43.05% (vanilla RAG)
Difficulty-adaptive RAG	LiveRAG	ACS, stratified by HD/D/M/E difficulty	System-specific, up to +80% on easy	Differentiates strengths/weaknesses across bins (Carmel et al., 18 Nov 2025)
SLA-driven agent reconfiguration	Production QA	F1, Hallucination Rate, Cost, Latency	F1 up to 0.683 (N=5); Halluc. 0.229	Linear cost scaling, diminished returns above N=5 (Iannelli et al., 7 Dec 2024)

LevelRAG demonstrates robust performance, matching or exceeding proprietary and open-source competitive baselines, and offers principled adaptation to dataset structure, operational constraints, and system-level requirements.

7. Limitations and Open Questions

A number of methodological and practical limitations have been identified:

• Summarization steps may prune critical facts, particularly for rare tokens (suggesting finer-grained fact extraction as a future direction) (Zhang et al., 25 Feb 2025).
• Current systems lack explicit adaptive stopping criteria for trivial queries, which could improve efficiency in heterogeneous settings.
• Cost and latency models require accurate offline performance curves for effective SLA-aware planning (Iannelli et al., 7 Dec 2024).
• Further integration of document-type or modality-specific retrievers and budget-aware query planning remains an open challenge.

A plausible implication is that incorporating multimodal retrieval and deeper query intent understanding may further strengthen LevelRAG's applicability in both research and production QA systems.