Progressive Retrieval Pipeline

Updated 3 April 2026

Progressive retrieval pipelines are multi-stage architectures that incrementally refine candidate results to balance retrieval accuracy and computational efficiency.
They utilize hierarchical embedding refinements, coarse-to-fine search, and progressive query expansion to reduce computational cost while preserving high precision.
Applications span large-scale RAG systems, scientific data management, and multimodal reasoning, achieving significant speedups and improved retrieval metrics.

A progressive retrieval pipeline is an architectural paradigm in information retrieval and data systems that incrementally refines candidate results, representations, or compressed data in multiple stages of increasing cost or fidelity. The approach is designed to optimize either computational efficiency, retrieval accuracy, retrieval latency, or precision-controlled data reconstruction, depending on the domain. Progressive retrieval pipelines have become central in large-scale retrieval-augmented generation (RAG) systems, cost-constrained retrieval tasks, scientific data management with demand-driven lossy compression, and multimodal reasoning, offering a balance between accuracy and efficiency unattainable via single-stage or “flat” retrieval paradigms (Jeong et al., 7 Feb 2026, Zhao et al., 2024, Yang et al., 6 Feb 2025).

1. Core Principles and Architectural Patterns

At the core of progressive retrieval lies staged refinement. The candidate set or representation begins with rapid, low-cost processing (usually lower-fidelity, reduced-dimension, or broader granularity), progressively narrowing down or refining candidates through increasingly computationally expensive or information-rich operations.

Key patterns include:

Hierarchical Search Over Embedding Dimensions: Start with embeddings projected into a low-dimensional subspace and, at each stage, increase the dimensionality for only a pool of candidates, culminating in full-dimension search for a minimal candidate set (Jeong et al., 7 Feb 2026).
Coarse-to-Fine Granularity: Operate retrieval at increasingly resolved granularity units—such as clusters → documents → passages—deploying more powerful models at later, smaller stages to improve ranking and context precision (Zhao et al., 2024).
Progressive Query or Evidence Expansion: Iteratively expand or refine the query using feedback from retrieved documents or evidence, typically combining pseudo-relevance feedback, LLM expansions, and relevance judging (Rashid et al., 2024, Yang et al., 2023).
Multilevel Data Decomposition: In scientific data systems, fields are encoded into progressive, bitplane-decomposed or multi-component representations from which reconstructions of arbitrary fidelity can be assembled by fetching only as much data as required by the client’s error tolerance (Yang et al., 6 Feb 2025, Magri et al., 2023).

This staged approach allows the system to minimize computational cost, latency, or data transfer while preserving or even improving top-k or top-1 accuracy.

2. Progressive Retrieval in RAG Systems

Modern RAG systems rely on vector retrieval over millions or billions of document embeddings to ground LLM responses.

Progressive Searching for RAG

The approach in "Progressive Searching for Retrieval in RAG" (Jeong et al., 7 Feb 2026) implements a multi-stage search architecture:

Stage 0: All document and query embeddings are truncated from full dimension $D$ to $D_s\ll D$ . A $K_0$ -nearest-neighbor search is performed on the entire database, yielding a candidate set $C$ .
Stage i: At each subsequent stage, double the search dimension $d_i \rightarrow d_{i+1}$ and halve (rounded down) the candidate count $K_i$ . The $K_{i+1}$ -NN search is performed on the candidate set in the new dimension, updating $C$ to the union of all distinct new candidates.
Final Stage: Perform a 1-NN search in the full embedding dimension, yielding the final result.

This process avoids scanning the full database in higher dimensions, resulting in significant speedups (up to 4.8× on 1M-document datasets, Table 1 and 2) while retaining essentially identical top-1 accuracy as full, non-progressive search at the same dimension (Jeong et al., 7 Feb 2026). The method relies only on truncation (no PCA or re-embedding) and supports both Euclidean and cosine similarity-based KNN.

FunnelRAG and Multi-Granular Pipelines

"FunnelRAG" (Zhao et al., 2024) demonstrates a coarse-to-fine progressive retrieval pipeline in RAG:

Coarse stage: Rapid sparse retrieval (e.g., BM25) of clusters (4K-token units) to retain top-K clusters from the full set.
Medium stage: Cross-encoder pre-ranking over document chunks within top clusters.
Fine stage: Passage-level scoring by Fusion-in-Decoder encoder–decoder, with selection based on averaged cross-attention weights, yielding final top-H passages.

This staged reduction, utilizing increasingly powerful (and costly) models at each candidate-narrowing stage, reduces latency by 31–43% (NQ and TQA datasets) while slightly improving Answer Recall and downstream QA Exact Match metrics. The progressive approach explicitly balances hardware constraints, retrieval granularity, and retriever capacity, overcoming the limitations of "flat" retrieval models (Zhao et al., 2024).

3. Algorithms and Theoretical Underpinnings

The mathematical and algorithmic foundations of progressive retrieval pipelines are characterized by hierarchical candidate reduction, dynamic adjustment of search fidelity, or adaptive query expansion.

Formalization of Progressive Search (in Embedding Retrieval)

Progressive search over embeddings is succinctly defined by projection:

$f_d(v) = (v_1,\ldots,v_d)^T$

At each stage, KNN operates in the truncated subspace, with top-K selection updating the candidate pool. There is no thresholding—pruning strictly follows the top-K rule. The time complexity is:

$\text{Single-stage:}~ O(ND) \qquad \text{Progressive:}~ O(ND_s + N_1d_1 + N_2d_2 + \dots + |C_{\text{final}}|D_{\max})$

with $D_s\ll D$ 0 and candidate pool sizes shrinking exponentially. The result is that the cost is dominated by the first low-dimensional scan (Jeong et al., 7 Feb 2026).

Progressive Query Expansion

ProQE (Rashid et al., 2024) refines the query by iterating:

Retrieve a single new document given the current query;
Use an LLM-based relevance judge and keyword extractor to update term weights;
Rebuild the intermediate query by boosting with positively judged terms, penalizing negatives;
Iterate until a hard cost or budget constraint is satisfied, then perform a final expansion (e.g., Chain-of-Thought).

This yields up to 37% higher MRR/R@1 over state-of-the-art baselines. Diminishing returns are observed beyond $D_s\ll D$ 1 expansion steps.

Staged Data Decomposition for Progressive Retrieval

Scientific data frameworks such as IPComp (Yang et al., 6 Feb 2025) and general multi-component schemes (Magri et al., 2023) encode the data into bitplanes or other progressive components, each contributing incrementally to reconstruction accuracy. The error bound is provably controlled by segment selection, with allocation and retrieval driven by user-specified fidelity or QoI constraints (see also (Wu et al., 2024)).

4. Domain-Specific Progressive Pipelines

Progressive Retrieval for Scientific Data

Bitplane or component-based representations underlie progressive data retrieval in scientific computing. IPComp (Yang et al., 6 Feb 2025) is a canonical example: data are bitplane-encoded post-interpolation prediction, grouped for efficient lossless coding. Retrieval is mediated by a dynamic-programming scheduler that optimally selects bitplanes/levels to satisfy error or bitrate constraints. This structure minimizes I/O and dramatically improves throughput (up to $D_s\ll D$ 2 faster than previous methods, $D_s\ll D$ 3 higher compression ratios).

Similarly, frameworks such as HP-MDR (Li et al., 1 May 2025) employ GPU-optimized pipelines for multi-level refactoring and bitplane encoding, including hybrid lossless codecs and pipeline overlap across host–device transfers. These stages admit on-the-fly reconstruction of primary or derived Quantities of Interest under strict error controls and achieve up to 10.4× kernel throughput improvements for QoI-driven reconstruction.

QoI-Error-Controlled Retrieval

"Error-controlled Progressive Retrieval of Scientific Data under Derivable Quantities of Interest" (Wu et al., 2024) extends progressive pipelines to guarantee user-specified error bounds on algebraic or composite derived quantities, not just on the raw data. Theoretical results show for any differentiable QoI $D_s\ll D$ 4,

$D_s\ll D$ 5

Offline, the data are transformed into progressive segments (e.g., bitplanes, wavelet coefficients, multi-snapshot deltas). Online, retrieval is staged with dynamic refinement: segments are fetched only as needed to ensure the reconstructed data's propagated error—computed via the full composition/basis bound structure—stays under the requested QoI tolerance, typically saving 2× or greater in data transfer costs compared to naive approaches (Wu et al., 2024).

5. Progressive Pipelines in Multimodal and Task Reasoning

"Progressive Evidence Refinement for Open-domain Multimodal Retrieval Question Answering" (Yang et al., 2023) establishes a two-stage system: (i) evidence retrieval via initial cosine-similarity filtering followed by BERT-based iterative inclusion, which sequentially constructs the logically-ordered chain of relevant evidence; (ii) question-answering through a multi-turn, multimodal dialogue with cross-attention fusion in a transformer—preserving fine-grained multimodal information.

Tool Retrieval and Task Decomposition

ProTIP (Anantha et al., 2023) presents progressive retrieval over compositional task spaces: at each step, the retrieval module updates a residual query embedding by vector subtraction of past-selected tool embeddings. This endows the retrieval process with subtask decomposition and state-tracking—without explicit textual rewriting or subtask labels—yielding a 24% increase in Recall@10 and a 41% improvement in tool accuracy for complex plan generation over SOTA decomposition-based retrieval.

6. Practical Integration, Hyperparameters, and Trade-offs

Progressive retrieval pipelines are characterized by a flexible interplay of hyperparameters controlling search cost vs. recall/accuracy:

Initial dimension ( $D_s\ll D$ 6) in progressive RAG search: Empirically, values in 64–256 balance cost and recall. Too small, and recall drops; too large, and the early scan dominates cost (Jeong et al., 7 Feb 2026).
Initial candidate pool ( $D_s\ll D$ 7): Should be large enough to likely include the true closest neighbors, but not so large as to make later stages expensive.
Number of stages and index structure: Indexes at each dimension can be brute-force or lightweight ANN (e.g., HNSW, IVF-PQ). Embedding matrices may be stored once and sliced for efficiency.
Complexity: Progressive pipelines generally incur total time near that of the first stage, with subsequent stages operating on small subsets; overall, this is orders of magnitude faster than naive high-dimensional search.
Trade-off: There is always a compromise between aggressiveness of early pruning and risk of recall loss; appropriately tuning hyperparameters yields north-west accuracy/latency curves (higher accuracy at lower latency).

For scientific data applications, component size or bitplane granularity controls the number/fidelity of retrieval increments; greedy or adaptive segment ordering may further optimize cost with respect to workload-specific downstream tolerances.

7. Empirical Evaluations and Comparative Results

The empirical superiority of progressive retrieval methods is evidenced by:

RAG Search: Up to 4.8× speedup with no statistically significant loss of top-1 accuracy compared to full-dimensional brute-force search (Jeong et al., 7 Feb 2026).
Coarse-to-fine pipelines: 31–43% latency reduction in retrieval modules, with simultaneous improvements in recall and downstream generation accuracy (Zhao et al., 2024).
Progressive query expansion: 37% MRR/R@1 gain over standard and LLM-only query-expansion baselines under cost constraints (Rashid et al., 2024).
Progressive scientific data retrieval: Up to $D_s\ll D$ 8 higher compression ratios, $D_s\ll D$ 9 faster throughput, and $K_0$ 0 end-to-end speedup in data transfer with strict error guarantees on user-defined quantities (Yang et al., 6 Feb 2025, Wu et al., 2024, Li et al., 1 May 2025).
Multimodal and planning scenarios: Substantial improvements in evidence and tool recall/accuracy without sacrificing architectural simplicity or requiring dense labels or text rewrites (Yang et al., 2023, Anantha et al., 2023).

The consistent pattern is that staged, adaptive, or residual-based progressive retrieval yields either large speedups, cost savings, or measurable accuracy improvements over both aggressive truncation and single-stage expensive search.

References:

“Progressive Searching for Retrieval in RAG” (Jeong et al., 7 Feb 2026)
“FunnelRAG: A Coarse-to-Fine Progressive Retrieval Paradigm for RAG” (Zhao et al., 2024)
“Progressive Query Expansion for Retrieval Over Cost-constrained Data Sources” (Rashid et al., 2024)
“IPComp: Interpolation Based Progressive Lossy Compression for Scientific Applications” (Yang et al., 6 Feb 2025)
“HP-MDR: High-performance and Portable Data Refactoring and Progressive Retrieval with Advanced GPUs” (Li et al., 1 May 2025)
“A General Framework for Progressive Data Compression and Retrieval” (Magri et al., 2023)
“Error-controlled Progressive Retrieval of Scientific Data under Derivable Quantities of Interest” (Wu et al., 2024)
“Progressive Evidence Refinement for Open-domain Multimodal Retrieval Question Answering” (Yang et al., 2023)
“ProTIP: Progressive Tool Retrieval Improves Planning” (Anantha et al., 2023)