ADORE: Iterative Query Expansion with Retrieval-Grounded Relevance Feedback

Abstract: LLM-based query expansion improves retrieval by enriching the original query with additional context. Yet most methods remain generation-driven, producing plausible pseudo-documents or expansions without checking how the target corpus responds. This can introduce retrieval drift, amplify misleading vocabulary, or miss terms that distinguish relevant from non-relevant documents. We argue that effective expansion requires retrieval-grounded feedback, not just single-pass generation or unverified iteration. We introduce ADORE (ADapt, Observe, Relevance Evaluate), an iterative framework that turns retrieval outcomes into feedback for the next expansion. At each round, an LLM generates pseudo-passages, a retriever exposes the corpus response, and a relevance assessor evaluates retrieved documents against the original query. These judgments identify what to reinforce, what remains undercovered, and what to suppress. Across TREC Deep Learning, BEIR, and BRIGHT, ADORE consistently outperforms strong query expansion baselines with notable improvements across nearly all evaluation settings, improving average nDCG@10 by 24.5% over BM25 and 3.6% over the strongest prior query expansion method on BEIR, and by 122.9% over BM25 and 9.2% over the best query expansion baseline on BRIGHT. Our code and data are publicly available.

• The paper presents a novel feedback-driven query expansion method that decouples term generation from relevance evaluation to mitigate retrieval drift.
• By iteratively incorporating LLM-based relevance feedback, ADORE achieves significant nDCG@10 improvements (24.5% to 122.9%) across diverse retrieval benchmarks.
• The approach enhances both sparse and dense retrieval systems efficiently, ensuring effective query reformulation in cross-domain scenarios.

Iterative Query Expansion with Retrieval-Grounded Feedback: An Analysis of ADORE

Introduction and Motivation

ADORE ("ADapt, Observe, Relevance Evaluate") proposes a retrieval-grounded approach for iterative query expansion in information retrieval (IR) systems using LLMs. While prior LLM-based query reformulation methods have demonstrated that synthetic expansions can improve retrieval, their reliance on generation-driven single-pass or unverified iterative procedures results in persistent limitations. Chief among these are the absence of retrieval-outcome awareness, shallow and unstructured use of corpus feedback, and empirical susceptibility to retrieval drift, especially on challenging and cross-domain benchmarks.

ADORE addresses these limitations by decoupling the expansion generation process from retrieval evaluation, instituting an explicit feedback loop in which each expansion is assessed via LLM-based relevance grading and the result is incorporated into subsequent reformulations. This structured, iterative process makes query reformulation retrieval-aware and corpus-calibrated, mitigating retrieval drift and improving recall of domain-specific terminology.

Methodology

ADORE formalizes query reformulation as an optimization problem: given an input query $q$, the goal is to construct expansions $\hat{q}$ that maximize a retrieval-specific evaluation metric $\mathcal{M}$ (e.g., nDCG@10) over a document corpus $\mathcal{C}$. Direct optimization is intractable due to the lack of ground-truth relevance at inference; thus, most prior methods resort to single-pass LLM generation or iterative generation using retrieved documents merely as prompt context. These approaches lack explicit diagnostics for the retrieval utility of expansion terms.

ADORE introduces an iterative framework comprising three stages per round:

1. Adapt: An LLM generates $n$ pseudo-passages conditioned on the original query and graded feedback from previous rounds. In the initial round, generation is zero-shot; in subsequent rounds, it is informed by accumulated feedback organized by relevance grade.
2. Observe: The generated passages, together with the original query, are used to construct a reformulated query, which is issued to a retriever (e.g., BM25), exposing the corpus ranking induced by the new query tokens.
3. Evaluate: An LLM-based relevance assessor assigns graded relevance labels (0–3) to newly retrieved documents with respect to the original query, not the expanded one. This feedback is stored, partitioned by grade, and drives the next generation round by highlighting terms to reinforce, undercovered signals, and off-target vocabulary to suppress.

The loop proceeds until an adaptive termination criterion is met—either a saturation in retrieval quality/coverage or a modest round limit, ensuring computational efficiency and robustness for harder queries.

Empirical Results

Benchmarks and Baselines

ADORE is evaluated on TREC Deep Learning, BEIR, and BRIGHT—datasets spanning passage, cross-domain, and reasoning-intensive retrieval tasks. Baselines include prominent LLM-based query expansion methods (GenQR, Query2Doc, ThinkQE, MUGI, LameR, CSQE), traditional techniques (BM25, RM3), dense retrievers (BGE, Contriever), and LLM-based rerankers.

Key Numerical Gains

• On BEIR: ADORE improves BM25 by 24.5% and outperforms the strongest query expansion baseline by 3.6% in average nDCG@10.
• On BRIGHT: ADORE surpasses BM25 by 122.9% and the next-best expansion method by 9.2%.
• Sparse-Dense Crossover: BM25 with ADORE expansions matches or exceeds the strongest dense retrievers (e.g., BGE-base-en-v1.5) on most settings, even though no retriever fine-tuning is used.

Case analyses show that ADORE effectively incorporates domain-specific lexical signals missed by zero-shot or single-pass expansions, for example elevating nDCG@10 from 0.610 to 0.916 in a passage retrieval task by iteratively integrating relevant terminology detected in prior corpus feedback.

Ablation studies confirm ADORE’s robustness to the choice of LLM reformulator or assessor, with negligible performance variance across models such as GPT-4.1, DeepSeek-V3, and Llama-3.3-70B. Iteration depth analysis suggests 2–3 rounds suffice for steep gains, with performance plateauing quickly, and adaptive early stopping minimizing unnecessary computation.

Notably, applying ADORE’s expansions to strong dense retrievers (BGE, Contriever) consistently boosts their own retrieval effectiveness, reaching nDCG@10 scores of 0.590 for BGE on BEIR (vs. 0.555 baseline).

Theoretical and Practical Implications

The decoupling of expansion generation from explicit relevance evaluation in ADORE sets a new regime for LLM-driven retrieval augmentation. The framework demonstrates that retrieval-grounded iterative feedback—anchoring assessment to the original intent and systematically organizing evidence—offers substantive advantages over purely generative or prompt-conditioned strategies, especially in cross-domain and reasoning-heavy retrieval scenarios.

Practically, ADORE lifts classical sparse retrievers (e.g., BM25) to near parity or superiority with respect to contemporary dense retrievers and even LLM-based rerankers without supervised retriever training. Its transferability to dense retrieval settings, where reformulated queries also yield significant improvements, highlights the approach's versatility.

Theoretically, ADORE motivates further research into the convergence properties of feedback-driven LLM loops, the integration of more sophisticated feedback mechanisms, and the impact of LLM biases on relevance grading.

Future Directions

Potential future avenues include:

• Latency Minimization: Given per-round LLM invocation overhead, developing approximations or hybrid early-stopping criteria for low-latency deployment remains important.
• Feedback Diversity: Incorporating alternative or learned feedback signals—e.g., from user interaction or unsupervised clustering—could further enhance expansion quality.
• Generalization Across Languages and Modalities: Extending ADORE to multilingual, domain-adaptive, or multimodal retrieval is a natural step, as is leveraging domain-specific LLMs for specialized corpora.
• Bias Mitigation: Exploring the impact of LLM bias in relevance grading and developing correction strategies is critical for robust deployment in underrepresented domains.

Conclusion

ADORE systematically advances the state of LLM-based query expansion by instituting a retrieval-grounded, feedback-driven loop that structurally mitigates retrieval drift and capture corpus-specific relevance signals. Empirical evaluation supports its consistent superiority across classic and challenging retrieval benchmarks, its model-component robustness, and its broad applicability to both sparse and dense retrieval paradigms. The iterative adaptation of query expansions to actual corpus behavior marks a significant methodological shift, with practical and theoretical implications for IR system design and the broader use of LLMs in interactive retrieval.

Reference: "ADORE: Iterative Query Expansion with Retrieval-Grounded Relevance Feedback" (2606.13905)