LURE-RAG: Utility-driven Reranking for Efficient RAG

• The paper introduces a lightweight reranking framework that directly optimizes retrieved passage order using a listwise LambdaMART loss to enhance generation performance.
• It leverages LLM-derived utility signals to score candidate documents, ensuring that the evidence selected maximizes token-level F1 and exact match metrics.
• Empirical evaluations show competitive accuracy with reduced inference latency and token overhead, making LURE-RAG a cost-effective choice for open-domain QA.

Lightweight Utility-driven Reranking for Efficient RAG (LURE-RAG) is a retrieval-augmented generation (RAG) framework that prioritizes generator-aligned utility, rather than strictly relevance-based criteria, for selecting and ranking evidence in open-domain question answering and related tasks. LURE-RAG leverages a lightweight reranker trained with a listwise ranking loss informed by LLM-derived utility signals, enabling its deployment atop any black-box retriever with minimal computational and engineering overhead. The core objective is to optimize the order and subset of retrieved passages such that downstream answer quality—measured by metrics such as token-level F1 or exact match—is maximized, while inference latency and resource requirements are minimized (Chandra et al., 27 Jan 2026, Song et al., 24 Jan 2026).

1. Motivation and Problem Statement

Traditional RAG pipelines operate in a three-stage manner: (1) a retriever generates a list of candidate passages, (2) the top-k candidates are concatenated as context with the query, and (3) an LLM produces a grounded answer. These retrieval models typically maximize relevance metrics (e.g., BM25, dense similarity), which have been shown to correlate only weakly—or even negatively—with downstream generator performance, especially under multi-passage evidence fusion. This disconnect arises because topically relevant but redundant or conflicting passages may harm the generator’s certainty or factuality (Song et al., 24 Jan 2026). As a result, existing approaches often fail to select the subset of retrieved documents most likely to enhance answer quality.

Utility-driven reranking reframes the selection problem: rather than relying on abstract notions of relevance, it quantifies each document's effect on an LLM’s likelihood of producing the ground-truth answer. LURE-RAG operationalizes this as a learning-to-rank problem, directly optimizing the ordering of documents for generation-specific utility.

2. LURE-RAG Architecture and Reranking Methodology

LURE-RAG consists of the following pipeline: (1) a black-box retriever returns N candidates for a query $q$; (2) a lightweight reranker scores each candidate using handcrafted and/or neural features; (3) candidates are ranked by reranker score; (4) the top $k$ are concatenated and fed to the generator. Crucially, the reranker is trained with utility-based supervision, ensuring that the ordering maximizes LLM answer quality (Chandra et al., 27 Jan 2026).

The reranker in the canonical LURE-RAG instantiation is a LambdaMART model—a gradient-boosted regression tree ensemble—that predicts a utility-aligned score for each $(q,d)$ pair, where $d$ is a candidate document. The training procedure is as follows:

1. For each query, candidate documents are assembled.
2. For each $(q,d)$ pair, the generator is prompted with $(q; d)$ in isolation to output an answer $\hat{y}_q(d)$.
3. A supervision signal is computed: $U(q, d) = P(y_q, \hat{y}_q(d))$, where $y_q$ is the gold answer and $P$ is either exact match or token-level F1.
4. Documents are ranked according to $U(q, d)$; the reranker is trained to predict this ordering, using a listwise LambdaMART loss that optimizes a surrogate for NDCG.
5. At inference, the reranker’s microsecond-level scoring enables selection of the most utility-promoting candidates to be presented to the LLM.

For domains where document features alone are insufficient, a dense variant (UR-RAG) extends this procedure using SBERT-based embeddings and a listwise neural ranker, still optimizing the utility-aligned loss (Chandra et al., 27 Jan 2026).

3. Formal Utility and Listwise Supervision

The utility signal in LURE-RAG is defined per-document as the downstream generator's answer quality. For each candidate $d$:

$U(q, d) = P(y_q, \hat{y}_q(d))$

where $P$ is a task-specific performance metric such as exact match or token-level F1. This differs fundamentally from relevance, as the utility may be low for textually relevant yet superfluous or distracting contexts.

The LambdaMART reranker receives as targets the set $\{U(q, d_i)\}$ for each query $q$ and learns to sort candidates to maximize cumulative NDCG (or any differentiable listwise objective that aligns closely with the desired ranking metric). The listwise loss penalizes misorderings of high-utility over low-utility documents, thereby directly shaping the evidence ranking to optimize generator outputs.

4. Feature Engineering and Lightweight Implementation

LURE-RAG’s reranker employs a 14-dimensional feature vector for each $(q,d)$ pair. The features include:

• Query and document statistics: length, number of distinct terms, min/max/mean IDF
• Query-document overlap: intersection size, BM25 score
• Topic-level features: cosine similarity between LDA topic distributions and top topic weights

LambdaMART models with $\approx$100 trees and depth $\leq$6 deliver extremely fast inference (microseconds per document) and seconds-level training on CPU. As the retriever and generator are kept black-box, no corpus re-encoding or index rebuilding is required, and LURE-RAG can be retrofitted onto existing pipelines (Chandra et al., 27 Jan 2026).

5. Empirical Evaluation: Accuracy, Efficiency, and Token Economy

LURE-RAG was evaluated on large-scale open-domain QA benchmarks (Natural Questions-Open, TriviaQA) using both sparse (BM25) and dense (Contriever) retrievers and multiple open-source LLMs. Quantitative results demonstrate:

• LURE-RAG achieves 97–98% of the F1 and accuracy of utility-driven dense retrievers (RePlug), with a much lower computational cost and no requirement to retrain or update retriever backbones.
• UR-RAG (dense SBERT variant) further improves over prior utility methods, with up to +3 F1 on certain LLM/dataset configurations.
• Feature ablations indicate BM25 similarity provides the dominant signal in sparse-only settings; topic and overlap features yield modest incremental gains.

For evidence selection under budget constraints, integration with information gain pruning methods results in +12–20% average F1 improvement and 76–79% reduction in input tokens—drastically boosting normalized token efficiency (Song et al., 24 Jan 2026).

Method	NQ Acc	NQ F1	TQA Acc	TQA F1
k-shot	0.288	0.159	0.284	0.260
RePlug	0.293	0.189	0.303	0.302
LURE-RAG	0.289	0.187	0.297	0.301
UR-RAG (dense)	0.300	0.207	0.311	0.310

6. Context, Variants, and Design Principles from Related Approaches

LURE-RAG is positioned within a family of recent utility-driven and lightweight reranking frameworks:

• SRAS uses a parameter-efficient policy network, trained under PPO with a hybrid reward (combining Relaxed-F1 and BERTScore), to maximize generation-aligned utility in document selection for edge and on-device settings (Muttur, 5 Jan 2026). This motivates similar principles in LURE-RAG: minimal model size, reward-driven training, and modular pipeline design.
• Information Gain Pruning (IGP) aligns evidence selection with generator uncertainty reduction. By quantifying the reduction in sequence-level entropy upon injecting a candidate passage, IGP prunes weak or harmful evidence before the budgeted truncate step, yielding major gains in token efficiency and final F1 (Song et al., 24 Jan 2026).
• InfoGain-RAG leverages a generator-driven “Document Information Gain (DIG)” signal to train a fast, utility-aware reranker, achieving even larger exact match gains and allowing for robust filtering of low-utility or negative-impact passages (Wang et al., 16 Sep 2025).
• Compressed-input reranking and pairwise prompting approaches achieve lightweight utility optimization by using document embeddings or extreme prompt efficiency, suggesting LURE-RAG can be further accelerated with modern compression or batching (Déjean et al., 21 May 2025, Wu et al., 10 Nov 2025).

7. Interpretability, Limitations, and Future Directions

The decision logic of LURE-RAG’s LambdaMART reranker is directly interpretable: tree structures expose splits on features (e.g., BM25, overlap, document length), and importance scores clarify which signals determine passage promotion or demotion. Qualitative inspections reveal that critical, answer-containing facts are systematically elevated in the reranked list, improving generator alignment (Chandra et al., 27 Jan 2026).

Limitations include the dependence on accurate proxy metrics for utility, which may require re-calibration or fine-tuning in new domains. Further, effectiveness is bounded by the retrieval candidate pool; utility-driven reranking cannot rescue fundamentally incomplete or adversarially misclassified retrieved sets. The pivotal ingredient for strong performance is the use of listwise, not pointwise or pairwise, ranking losses—this aligns the candidate ordering closely with generation behavior (Chandra et al., 27 Jan 2026).

Potential extensions include integrating dynamic and compressed feature architectures, adding feedback from real-world user interactions, or applying information gain pruning adaptively as part of the reranking budget (Song et al., 24 Jan 2026, Déjean et al., 21 May 2025). The plug-and-play design and black-box compatibility position LURE-RAG as a strong candidate for wide deployment in efficient, generator-aligned RAG pipelines.