LLM-Based Retrieval Strategy

• LLM-based Retrieval Strategy is a paradigm that uses large language models to drive query understanding, concept extraction, and adaptive reranking in information systems.
• It integrates modular stages such as semantic index construction, embedding matching, and listwise reranking to boost metrics like recall and nDCG over traditional methods.
• Applications in scientific literature search and complex QA demonstrate its practical benefits in efficiency, scalability, and robust, interpretable retrieval.

A LLM-based retrieval strategy denotes retrieval paradigms and architectures in which LLMs directly inform, control, or parameterize one or more core retrieval stages—query understanding, document encoding, concept selection, reranking, or evidence verification—within information access systems. Unlike traditional retrieval that relies primarily on lexical or shallow neural approaches, LLM-based retrieval processes leverage the semantic, generative, and reasoning capabilities of large pretrained LLMs to enable high-granularity, robust, and interpretable retrieval, especially in scientific, complex QA, and structured domains.

1. Core Principles and Architectures of LLM-Based Retrieval

LLM-based retrieval strategies typify a set of principles that shift the control of relevance, representation, and ranking from brittle lexical signals to corpus- and query-aware LLM outputs. The canonical LLM-based retrieval pipeline may encompass several (possibly modular) stages:

• LLM-Guided Query Understanding: LLMs are prompted with user queries, candidate documents, and corpus-derived semantic units to extract or select core query concepts (e.g., key topics, technical phrases) in a faithful, grounded manner, mitigating hallucination and aligning search intent with the corpus’ ontology (Zhang et al., 27 May 2025).
• Semantic Index Construction: Each document is pre-indexed by concepts at multiple granularities—including general research topics and fine-grained key phrases—using LLM-based extraction or classification, with both symbolic and embedding representations.
• Embedding Matching and Soft Scoring: At query time, relevance is computed via embedding-based soft matching between the LLM-selected core concepts and each document’s indexed concept vectors, often fused with base scores from traditional retrievers.
• Listwise and Adaptive Reranking: LLMs may operate as high-capacity listwise rerankers, consuming compressed (or full-text) document features, or reranking adaptively via feedback-driven retrieval windows and graph expansion (Tian et al., 19 May 2025, Rathee et al., 15 Jan 2025).
• Iterative/Verifiable Refinement: Higher-level control loops interleave retrieval with LLM-based verification, dynamically updating candidate pools until sets are verified to fully support the information need (Li et al., 2023).

Contemporary frameworks such as SemRank (Zhang et al., 27 May 2025), CoRank (Tian et al., 19 May 2025), SlideGar (Rathee et al., 15 Jan 2025), LMORT (Sun et al., 2024), and LLatrieval (Li et al., 2023) exemplify these principles, with architectural choices tailored to specific domains and performance constraints.

2. Concept Extraction, Semantic Indexing, and Prompt Design

The efficacy of LLM-based retrieval depends on precise extraction and structured representation of scientific concepts and document features:

• Multigranular Concept Spaces: Documents are indexed with both general topics (drawn from taxonomies such as MAG) and key phrases auto-extracted from titles and abstracts. Each concept is encoded both as a discrete label and as a point in the retriever’s embedding space (e.g., SPECTER-v2) (Zhang et al., 27 May 2025).
• Document Feature Extraction: Zero-shot or few-shot LLMs are leveraged to extract compact document features (categories, section headings, keywords, pseudo-queries) for high-coverage candidate selection under context budget constraints (Tian et al., 19 May 2025).
• Prompt Engineering: LLMs are prompted via templates to select from candidate concept pools—converted from generation to selection tasks—or to produce compressed document representations suitable for massive listwise reranking. Carefully constraining prompts to corpus-specific vocabularies dramatically curbs hallucination while enhancing relevance fidelity (Zhang et al., 27 May 2025).

The following table summarizes key prompt types and their functional roles:

Prompt Type	Context Provided	Output/Function
Query Concept Extraction	Query, top-k abstracts, candidate concepts	<ans> best-matching concepts
Document Indexing	Abstract, candidate topics	<top> topic list; <kp> key phrase list
Coarse Reranking	Query, 200 feature summaries	Permutation (ordering) of candidate docs
Verification/Selection	Query, candidate pool	Best k docs or setpass/fail (Yes/No)

3. Mathematical Formulations and Algorithmic Workflows

LLM-based retrieval strategies instantiate concrete mathematical formulations for concept matching, semantic ranking, and iterative search. Key formulas and steps include:

• Semantic Concept Matching:

$$s^\text{sem}(q, d_i) = \frac{1}{|C(q)|} \sum_{c \in C(q)} \max_{c' \in C_i} \cos(\mathbf{c}, \mathbf{c}')$$

where $C(q)$ is the set of LLM-extracted core concepts for query $q$, and $C_i$ is the indexed concept set for document $d_i$ (Zhang et al., 27 May 2025).

• Final Retrieval Score:

$$s(q, d_i) = z(s^\text{base}(q, d_i)) + z(s^\text{sem}(q, d_i))$$

using z-score normalization (Zhang et al., 27 May 2025).

• Coarse-to-Fine Reranking:

Given $m$ candidate docs, stage-1 scoring is

$$s_i^{\text{feat}} = \text{ScoreLLM}(q, r_i; [r_1, ..., r_m])$$

and the top $k$ are rescored in stage-2 on their full text (Tian et al., 19 May 2025).

• Adaptive/Iterative Retrieval (SlideGar): Alternates LLM listwise ranking and graph-based candidate pool expansion to overcome bounded recall, keeping total LLM inference calls constant (Rathee et al., 15 Jan 2025).

4. Comparative Empirical Performance

LLM-based retrieval strategies deliver substantial gains in recall, nDCG, and citation-F1 across diverse scientific and open-domain benchmarks:

• SemRank yields Recall@5/20/100 (LitSearch): from 0.393/0.555/0.720 (SPECTER-v2 alone) to 0.503/0.632/0.775, a 28–44% relative improvement (Zhang et al., 27 May 2025).
• CoRank increases nDCG@10 from 32.0 to 39.7 averaged across LitSearch/CSFCube using compact feature reranking (Tian et al., 19 May 2025).
• Listwise Adaptive Reranking (SlideGar) improves nDCG@10 by up to 13.2% and recall by 28%, with no extra LLM cost (Rathee et al., 15 Jan 2025).
• Verification-driven frameworks (LLatrieval) achieve new SOTA citation-F1 and correctness on multi-evidence QA datasets, e.g., ASQA Cite-F1 61.1% vs. 57.5% (baseline) (Li et al., 2023).

Table: Summarized Empirical Results

Method	Metric	Baseline	+ LLM-based Retrieval	Relative Improvement
SemRank	Recall@20	0.555	0.632	+14%
CoRank	nDCG@10	32.0	39.7	+24%
SlideGar	Recall@c (D19)	0.389	0.498	+28%
LLatrieval	Cite-F1 (ASQA)	57.5	61.1	+6%

5. Efficiency, Scalability, and Design Considerations

Several strategies reconcile tight compute budgets with high retrieval quality:

• Lightweight LLM Calls: Approaches such as SemRank require only one LLM call per query (average output ≈ 19 tokens, ≈ 1.8 s latency), running entirely with CPU-based embedding matching (Zhang et al., 27 May 2025).
• Compact Feature Reranking: Feature-based passes allow reranking of up to 200 candidates in a single LLM window, sidestepping context window limitations (Tian et al., 19 May 2025).
• Post-hoc Fusion and Adaptive Expansion: Graph-based and multi-source feedback-driven approaches permit coverage expansion without increasing LLM inference cost (Rathee et al., 15 Jan 2025).
• Plug-and-Play Modularity: These frameworks are designed to wrap around off-the-shelf dense/sparse retrievers, requiring neither retriever retraining nor query supervision (Zhang et al., 27 May 2025).

6. Limitations and Future Directions

Current LLM-based retrieval architectures are constrained by several factors:

• Inter-Concept Relation Blindness: Most approaches index and match topics/phrases independently, neglecting hierarchical or graph-structured concept relations that could further refine semantic matching (Zhang et al., 27 May 2025).
• Partial Corpus Coverage: Indexing is typically limited to titles and abstracts to constrain LLM prompt size, omitting supplementary material, citations, and full text rich in latent concepts (Zhang et al., 27 May 2025).
• Prompt Sensitivity: Quality of LLM-derived features and selection is sensitive to prompt design and may not generalize across domains or languages without adaptation (Zhang et al., 27 May 2025, Tian et al., 19 May 2025).
• Scalability and Latency: Ultra-large first-stage candidate pools and massive corpora may entail significant memory requirements or necessitate distributed inference for feasible latency at scale (Tian et al., 19 May 2025).

Research questions addressed in recent and ongoing work include:

• Construction and exploitation of dynamic concept graphs (topics ↔ phrases), possibly via GNNs.
• Joint learning of concept embedding and topic classifiers for robust zero-shot transfer.
• Full-paper indexing tradeoffs vis-à-vis context budget and LLM inference costs.
• Cross-lingual and multi-modal extensions.

7. Synthesis and Research Significance

LLM-based retrieval strategies have redefined standard assumptions regarding the granularity, interpretability, and reliability of scientific paper and open-domain document search. By explicitly coupling LLM-driven query understanding, faithful multi-granular indexing, and efficient hybrid scoring, these methods establish substantial empirical advantages over baseline dense and lexical retrievers across recall, nDCG, and verifiability. Robustness to initial ranking quality, ability to adaptively expand or compress candidate sets, and interpretability via explicit concept matching or feature extraction distinguish these paradigms. A plausible implication is that LLMs, when systematically incorporated into both query and document understanding, fundamentally elevate retrieval to semantically faithful, corpus-aligned, and efficiency-conscious reasoning tasks, forming the backbone for next-generation literature discovery and information access (Zhang et al., 27 May 2025, Tian et al., 19 May 2025, Rathee et al., 15 Jan 2025, Li et al., 2023).