NQ_Rerank: LLM Confidence-Based Reranking

• NQ_Rerank is a preference-based reranking dataset designed to align candidate contexts with the LLM's confidence shifts for improved answer utility.
• It leverages a hidden-state confidence detector trained on Natural Questions to convert internal model signals into positive and negative context preferences.
• The approach supports dynamic retrieval and demonstrates enhanced precision, recall, and MRR metrics compared to traditional semantic reranking methods.

NQ_Rerank is a preference-style reranking dataset built from Natural Questions to align a reranker with the actual downstream LLM’s preferences rather than with generic semantic relevance. Its central supervision signal is whether a candidate retrieved context increases or decreases the target LLM’s estimated confidence that it can answer the question correctly. In the underlying framework, a hidden-state confidence detector is trained on the target model’s internal representations, and those confidence shifts are then converted into positive and negative context preferences for reranker fine-tuning and for confidence-based dynamic retrieval (Jin et al., 8 Sep 2025).

1. Purpose and conceptual framing

NQ_Rerank was created to address a mismatch that arises in retrieval-augmented generation: standard rerankers are usually trained on semantic relevance labels, so they optimize whether a context “matches” the query, not whether it actually helps a particular LLM answer. In this formulation, a passage is useful if it increases the LLM’s confidence that it can answer correctly; a merely topically related passage may be neutral or even harmful. NQ_Rerank is therefore designed for post-retrieval filtering, where higher ranks should be assigned to contexts that increase model confidence and lower ranks to contexts that reduce it (Jin et al., 8 Sep 2025).

This design is closely related to a broader critique of static relevance supervision in RAG. A related line of work argues that documents identified as topically relevant by information retrieval metrics often fail to provide the actual utility required by the LLM for precise answer generation, and instead optimizes reranking directly with LLM feedback on answer quality (Wu et al., 2 Apr 2026). This suggests that NQ_Rerank belongs to a class of answer-utility-oriented reranking methods rather than conventional semantic-similarity reranking.

2. Hidden-state confidence estimation

NQ_Rerank depends on a prerequisite model: a confidence detector that reads the target LLM’s internal hidden states. For target LLM $M$ and question $Q$, the method extracts the hidden state at the mid layer, i.e. Layer/2, before generating the first answer token. This hidden state is denoted $H_{M,Q}$. A classifier $E$ is then trained to predict whether the LLM will answer correctly:

$C_{M,Q} = E(H_{M,Q})$

where $C_{M,Q}=1$ means the model is confident it can answer correctly and $C_{M,Q}=0$ means it is not. The classifier is then used as a probability-valued confidence estimator, and that confidence becomes the supervision source for reranking (Jin et al., 8 Sep 2025).

The detector is trained from Natural Questions question-answer examples. For each question, the target LLM is run, the mid-layer pre-first-token hidden state is collected, and the model’s answer is compared to the ground truth to produce binary labels. The reported split sizes for this confidence detector are 1,000 positive and 1,000 negative training examples, 300 positive and 300 negative development examples, and 500 positive and 500 negative test examples. The training details reported for $E$ are learning rate $5 \times 10^{-5}$, dropout $0.5$, and 30 epochs. The primary downstream LLM whose preferences define the dataset is Llama3-8B-Instruct, and the paper also evaluates transfer to Qwen2.5-7B-Instruct (Jin et al., 8 Sep 2025).

3. Construction of the dataset

Once the confidence detector has been trained, NQ_Rerank is generated by measuring how each retrieved context changes the LLM’s confidence. For a question $Q$0 with candidate contexts $Q$1, the method computes hidden states for the query alone, $Q$2, and for the query paired with each candidate context, $Q$3. These are passed through the detector to obtain confidence estimates, and the utility of a context is defined as the confidence increase:

$Q$4

If $Q$5, the context is positively preferred; if $Q$6, it is negatively preferred. All candidate contexts for a query are ranked by $Q$7, the Top-5 highest confidence-increasing contexts are selected as positive examples, and the Top-5 largest confidence-decreasing contexts are selected as negative examples. Queries lacking at least one valid positive and one valid negative are filtered out. The resulting labels are therefore not human judgments and not direct passage-level correctness labels; they are LLM-preference labels induced by internal confidence changes (Jin et al., 8 Sep 2025).

Artifact	Split or setting	Value
Confidence detector	Train	1,000 positive + 1,000 negative
Confidence detector	Dev	300 positive + 300 negative
Confidence detector	Test	500 positive + 500 negative
NQ_Rerank	Training samples	7,622
NQ_Rerank	Evaluation samples	1,216
NQ_Rerank	Positives or negatives per query	1 to 5

All reranker supervision in the paper is derived from NQ only. The retriever is intentionally held fixed and excluded from comparison so that every reranker sees the same set of retrieved contexts. Operationally, the dataset originates from sorting a candidate list by confidence increase, but the training consumption is contrastive: it is closest to pairwise or one-positive-vs-many-negatives contrastive ranking rather than to a full listwise objective (Jin et al., 8 Sep 2025).

4. Fine-tuning and end-to-end pipeline

The reranker fine-tuned on NQ_Rerank is bge-reranker-v2-m3, a 568M-parameter model. Other rerankers used as baselines are gte_passage-ranking_multilingual-base, Qwen3-Reranker-4B, Qwen3-Reranker-8B, and the unfine-tuned bge-reranker-v2-m3. The fine-tuned system is denoted bge-reranker-v2-m3-ft (Ours). The reranker computes a score $Q$8 written as

$Q$9

with temperature parameter $H_{M,Q}$0, and is optimized by an InfoNCE objective:

$H_{M,Q}$1

where $H_{M,Q}$2 is a positive context and $H_{M,Q}$3 are negative contexts. For fine-tuning bge-reranker-v2-m3 on NQ_Rerank, the reported hyperparameters are learning rate $H_{M,Q}$4, weight decay $H_{M,Q}$5, max query length 128, max passage length 512, and 1 epoch (Jin et al., 8 Sep 2025).

The full system has four stages. First, a hidden-state confidence detector is trained from NQ question-answer examples. Second, post-retrieval confidence preferences are generated by comparing confidence on the query alone with confidence on the query plus each candidate context. Third, the reranker is fine-tuned on these preferences so that it scores contexts that increase LLM confidence above contexts that decrease it. Fourth, the same confidence detector is also used for Confidence-Based Dynamic Retrieval (CBDR): if $H_{M,Q}$6, retrieval and reranking are skipped and the model answers directly; if $H_{M,Q}$7, retrieval, reranking, and generation are executed (Jin et al., 8 Sep 2025).

5. Evaluation and reported results

On the NQ_Rerank test set, rerankers are evaluated by whether positive contexts are placed near the top. The reported metrics are Precision@K, Recall@K, and MRR@K at $H_{M,Q}$8, motivated by the fact that each query has between 1 and 5 positive or negative contexts. The paper gives the MRR formula as

$H_{M,Q}$9

where $E$0 is the rank of the first relevant context for query $E$1, truncated beyond $E$2 (Jin et al., 8 Sep 2025).

The best reported model on NQ_Rerank is bge-reranker-v2-m3-ft. Its reported results are Precision@1 $E$3, Recall@1 $E$4, and MRR@1 $E$5; Precision@3 $E$6, Recall@3 $E$7, and MRR@3 $E$8; Precision@5 $E$9, Recall@5 $C_{M,Q} = E(H_{M,Q})$0, and MRR@5 $C_{M,Q} = E(H_{M,Q})$1. Relative to the unfine-tuned bge-reranker-v2-m3 baseline, Precision@1 and MRR@1 improve by $C_{M,Q} = E(H_{M,Q})$2 percentage points and Recall@1 improves by $C_{M,Q} = E(H_{M,Q})$3 percentage points. Relative to Qwen3-Reranker-8B, Precision@1 and MRR@1 improve by $C_{M,Q} = E(H_{M,Q})$4 percentage points and Recall@1 improves by $C_{M,Q} = E(H_{M,Q})$5 percentage points (Jin et al., 8 Sep 2025).

The paper also reports downstream RAG evaluation. With Llama3-8B-Instruct as the generator, the NQ_Rerank-trained reranker improves end-to-end performance. On NQ, bge-reranker-v2-m3 gives Top-1 $C_{M,Q} = E(H_{M,Q})$6 and Top-3 $C_{M,Q} = E(H_{M,Q})$7, whereas bge-reranker-v2-m3-ft gives Top-1 $C_{M,Q} = E(H_{M,Q})$8 and Top-3 $C_{M,Q} = E(H_{M,Q})$9. On HotpotQA with the same generator, bge-reranker-v2-m3 gives Top-1 $C_{M,Q}=1$0 and Top-3 $C_{M,Q}=1$1, whereas bge-reranker-v2-m3-ft gives Top-1 $C_{M,Q}=1$2 and Top-3 $C_{M,Q}=1$3. By contrast, gains are negligible for Qwen2.5-7B-Instruct, and the authors interpret this as evidence that reranker preference alignment is partly LLM-specific (Jin et al., 8 Sep 2025).

For CBDR on NQ, the paper reports that without dynamic retrieval, bge-reranker-v2-m3-ft yields Top-1 $C_{M,Q}=1$4 and Top-3 $C_{M,Q}=1$5. With $C_{M,Q}=1$6, it yields Top-1 $C_{M,Q}=1$7, Top-3 $C_{M,Q}=1$8, and retrieval overhead saved $C_{M,Q}=1$9. With $C_{M,Q}=0$0, it yields Top-1 $C_{M,Q}=0$1, Top-3 $C_{M,Q}=0$2, and retrieval overhead saved $C_{M,Q}=0$3. The presentation is numerically inconsistent with the abstract, which reports a 7.10% reduction in retrieval costs while maintaining 5.60% accuracy gains; the detailed table instead shows very large skip rates and modest accuracy movement depending on threshold and top-$C_{M,Q}=0$4 setting (Jin et al., 8 Sep 2025).

6. Interpretation, limitations, and relation to adjacent work

The preferred-context criterion in NQ_Rerank is explicitly answer-enabling rather than merely topically similar. The paper’s illustrative example is the question “who played karen in married to the mob?” A weak but semantically related context mentions that Married to the Mob starred Michelle Pfeiffer and Matthew Modine; a more useful context explicitly states that Karen (Nancy Travis) was Tony Russo’s mistress. The second context raises answer confidence and is therefore preferred. This clarifies that the dataset is designed to reflect utility to a specific LLM, not generic relevance (Jin et al., 8 Sep 2025).

Several limitations are also built into the dataset construction. First, NQ_Rerank is LLM-specific: because its labels come from confidence shifts of a particular target LLM, transfer may be weak for other generators. Second, all labels depend on the confidence detector $C_{M,Q}=0$5; if $C_{M,Q}=0$6 misestimates confidence, the preference labels are noisy. Third, filtering out queries without clear positive and negative contexts introduces selection bias toward examples with strong confidence gradients. Fourth, all training supervision comes from NQ, so the learned preference structure may reflect NQ-style factoid QA more than broader tasks. Fifth, there is no human validation that confidence-based labels perfectly match utility (Jin et al., 8 Sep 2025).

Related reranking studies reinforce both the rationale and the cautions around this formulation. Reinforcement-learning-based reranking work likewise argues that documents identified as topically relevant by standard IR metrics often fail to provide the actual utility required by the LLM for precise answer generation, and instead optimizes passage ordering directly for answer quality (Wu et al., 2 Apr 2026). Conversely, an analysis of LM rerankers on NQ shows that they still fail systematically on examples where the gold passage is lexically dissimilar to the query or where a non-gold distractor has stronger lexical overlap, and finds that prepending page titles yields the greatest effects on NQ (Hagström et al., 24 Feb 2025). This suggests that NQ_Rerank’s confidence-shift supervision addresses one axis of misalignment—answer utility—while lexical confusability and missing document context remain separate sources of reranking error.