Selective Evidence-Grounded Finalizer

• The paper demonstrates that the selective evidence-grounded finalizer verifies candidate answers using a three-way decision process (Supported, Refuted, Insufficient) to ensure evidence sufficiency.
• It decomposes candidate answers into individual claims and aggregates pair-level signals from evidence passages using a shared verifier to assess overall sufficiency.
• This approach improves low-coverage selective answering by significantly reducing unsafe answers compared to traditional pooling-based methods.

Searching arXiv for the cited paper and closely related selective RAG verification work. Selective Evidence-Grounded Finalizer denotes a retrieval-augmented generation (RAG) component that does not treat retrieval as proof, but instead verifies whether the retrieved evidence is actually sufficient to justify a candidate answer, and answers only when support is established. In "SURE-RAG: Sufficiency and Uncertainty-Aware Evidence Verification for Selective Retrieval-Augmented Generation" (Qiu et al., 5 May 2026), this role is instantiated as a three-way evidence sufficiency verifier over $(q,a,E)$, where $q$ is a question, $a$ is a candidate answer, and $E=\{e_1,\dots,e_k\}$ is the retrieved evidence set. The central premise is that evidence sufficiency is a set-level property: topical passages may still fail to justify an answer because of missing hops, partial support, or unresolved conflicts, so selective answering requires verification rather than retrieval relevance alone (Qiu et al., 5 May 2026).

1. Task semantics and selective answering

The paper formulates evidence sufficiency verification as prediction of

$y \in \{Supported, Refuted, Insufficient\}$

from $(q,a,E)$ (Qiu et al., 5 May 2026). The semantics are explicit. Supported means that the evidence entails the central factual content of $a$. Refuted means that the evidence contradicts at least one central factual claim in $a$. Insufficient means that the evidence cannot justify $a$, including missing hops, partial support, topical-but-not-sufficient passages, and unresolved conflicts.

This three-way decision is converted into selective answering by the mapping:

• Supported $\rightarrow$ Safe / Answer
• Refuted or Insufficient $q$0 Unsafe / Abstain

Within this formulation, the verifier is not merely a classifier. It is a reject-option verifier that abstains unless the evidence is sufficiently strong (Qiu et al., 5 May 2026). The paper defines the accepted set under a thresholded selective score $q$1 as

$q$2

with

$q$3

These definitions make selective answering operational in terms of both how often the system answers and how often those answers are unsafe (Qiu et al., 5 May 2026).

A plausible implication is that the notion of “finalizer” is narrower than end-to-end answer generation. The component is positioned after retrieval and candidate-answer production, and its role is to authorize or block the answer on evidence-grounded sufficiency criteria.

2. Shared pair-level verification and answer decomposition

SURE-RAG has three components: pair-level claim-evidence verification, answer-level sufficiency aggregation, and selective answering (Qiu et al., 5 May 2026). The answer $q$4 is decomposed into claims $q$5. For each claim $q$6 and evidence passage $q$7, a shared DeBERTa-v3-base cross-encoder produces a local relation distribution,

$q$8

corresponding to supported, refuted, and neutral (Qiu et al., 5 May 2026).

The use of a shared pair verifier across SURE-RAG and the pooling baselines is methodologically important because any gain from SURE-RAG comes from the aggregation logic, not from a stronger encoder. The paper also notes that for long-form answers it extends FActScore-style atomic claim decomposition, while the main benchmark focuses on short-answer multi-hop verification (Qiu et al., 5 May 2026).

This design separates local relational judgments from answer-level sufficiency. A single claim-passage pair can be locally supportive or refutational, yet the full answer may still be insufficient if another required claim is uncovered or if the evidence collection contains unresolved contradiction. This suggests that decomposition is not only a transparency device but also a structural precondition for set-level verification.

3. Set-level sufficiency aggregation and interpretable signals

The paper’s key idea is that sufficiency is a set-level property of the evidence collection, so pairwise scores must be aggregated into answer-level evidence structure (Qiu et al., 5 May 2026). SURE-RAG constructs an answer-level feature vector $q$9 from the set $a$0, then predicts

$a$1

where $a$2 is a lightweight classifier over the aggregated features.

The aggregation produces five kinds of signals:

Signal block	Features or role
Coverage	supported claim fraction; refuted claim fraction; insufficient claim fraction
Relation strength	max support; max refutation; mean neutral
Disagreement	dispersion-style signal derived from pair-level distributions
Conflict	support-refute contradiction across passages
Retrieval uncertainty	optional retrieval-side signal from retriever or retrieval metadata

Coverage is derived by per-claim argmax over passage-level labels. Relation strength captures whether any passage provides a strong local signal. Disagreement measures internal inconsistency in the evidence collection. Conflict is intended to detect unresolved contradiction among retrieved passages. Retrieval uncertainty is optional and can be removed in no-oracle tests to check whether retrieval metadata itself is acting as a shortcut (Qiu et al., 5 May 2026).

The paper’s rationale for these blocks is sharply contrasted with simpler pooling. Max pooling over-commits if one passage looks supportive but another hop is missing; mean pooling dilutes strong refutation or support; sufficiency requires understanding coverage, conflicts, and uncertainty across the full set (Qiu et al., 5 May 2026). This suggests that the “finalizer” is fundamentally an aggregation protocol rather than just a claim-passage scorer.

4. Decision rule, uncertainty penalty, and calibration

The pipeline outputs a three-way class distribution $a$3, and chooses

$a$4

For selective answering, SURE-RAG defines the selective score

$a$5

where $a$6 is $a$7, $a$8 is an uncertainty penalty, and $a$9 is a risk weight (Qiu et al., 5 May 2026). The uncertainty penalty combines predictive entropy, evidence disagreement, conflict, coverage deficit, and optional retrieval uncertainty.

The final decision rule is:

1. Decompose $E=\{e_1,\dots,e_k\}$0 into claims $E=\{e_1,\dots,e_k\}$1
2. Compute all pairwise $E=\{e_1,\dots,e_k\}$2
3. Aggregate into $E=\{e_1,\dots,e_k\}$3
4. Predict $E=\{e_1,\dots,e_k\}$4
5. Set $E=\{e_1,\dots,e_k\}$5
6. Compute $E=\{e_1,\dots,e_k\}$6
7. Answer iff $E=\{e_1,\dots,e_k\}$7 and $E=\{e_1,\dots,e_k\}$8

This rule means that even if the classifier predicts Supported, the system can still abstain when uncertainty is too high (Qiu et al., 5 May 2026). Post-hoc calibration parameters, threshold $E=\{e_1,\dots,e_k\}$9, and risk weight $y \in \{Supported, Refuted, Insufficient\}$0 are all selected on the development split, with no test labels used. The calibrated variant uses the same mechanism but improves calibration metrics (Qiu et al., 5 May 2026).

A plausible implication is that the selective score functions as an auditable gate: the answer is conditioned not only on the class label but also on a structured uncertainty estimate whose components are individually interpretable.

5. Benchmark construction and artifact-aware evaluation

The primary benchmark is HotpotQA-RAG v3, derived from HotpotQA’s multi-hop structure. It contains 4026 examples and 900 question groups, with group-disjoint train/dev/test splits by original question ID (Qiu et al., 5 May 2026). The benchmark includes five evidence conditions:

1. Full — all supporting facts plus distractors; Label: Supported
2. Partial — one necessary supporting fact removed; Label: Insufficient
3. Hard-insufficient — topically related/high overlap but missing required relation; Label: Insufficient
4. Irrelevant — only distractors; Label: Insufficient
5. Naturally refuted — answer is perturbed while original evidence remains, so evidence contradicts answer; Label: Refuted

A notable detail is that pair labels are derived from construction metadata, not copied from answer labels. The benchmark yields 20,130 claim-evidence pairs total, with 3292 support, 2202 refute, and 14636 neutral (Qiu et al., 5 May 2026).

The evaluation protocol is explicitly artifact-aware. It includes shortcut baselines, counterfactual evidence swaps, no-oracle checks, GPT-4o semantic audits, risk-coverage metrics, and external transfer on HaluBench (Qiu et al., 5 May 2026). The shortcut baselines are majority, hypothesis-only, evidence-only, length-only, overlap-only, and concat TF-IDF. The no-oracle checks remove retrieval scores or replace them with BM25. Counterfactual evidence swaps compare $y \in \{Supported, Refuted, Insufficient\}$1 on full versus degraded evidence for the same $y \in \{Supported, Refuted, Insufficient\}$2. This is presented as a major part of the contribution: not just benchmark performance, but robustness against artifacts.

6. Empirical results and task boundary with hallucination detection

On HotpotQA-RAG v3, the reported Macro-F1 results are:

Method	Macro-F1
SURE-RAG	0.8951 $y \in \{Supported, Refuted, Insufficient\}$3
SURE-RAG calibrated	0.9075 $y \in \{Supported, Refuted, Insufficient\}$4
DeBERTa max-pool	0.5948
DeBERTa mean-pool	0.6516
DeBERTa top-$y \in \{Supported, Refuted, Insufficient\}$5	0.6444
Concat cross-encoder	0.8888
SURE pooling-only	0.8383

These results show that SURE-RAG decisively beats all pooling baselines, matches or slightly exceeds the strong concat cross-encoder, and remains auditable because it exposes decomposition and aggregation signals (Qiu et al., 5 May 2026).

For calibration, the paper reports binary ECE on safe/unsafe of 0.0304 for raw SURE-RAG and 0.0198 for calibrated SURE-RAG (Qiu et al., 5 May 2026). For selective answering, the low-coverage regime is the clearest strength: at Risk@30, max-pool is 0.3183, mean-pool is 0.2588, SURE-RAG is 0.1642, and calibrated SURE-RAG is 0.1670, corresponding to a 37% reduction in unsafe answers relative to mean-pooling at 30% coverage (Qiu et al., 5 May 2026). At higher coverage, the advantage shrinks: at Risk@50, SURE-RAG is 0.3600 and mean-pool is 0.3783; at Risk@70, SURE-RAG is 0.5457 and mean-pool is 0.5350. The stated conclusion is therefore nuanced: SURE-RAG clearly helps in low-coverage selective ranking, but it does not solve high-coverage risk control.

The no-oracle checks report 0.8370 Macro-F1 when retrieval score is dropped and 0.8347 Macro-F1 when BM25 retrieval score is used; both remain above DeBERTa mean-pooling, so retrieval metadata is not the sole driver (Qiu et al., 5 May 2026). Counterfactual swap sensitivity is near-perfect: full versus partial, hard-insufficient, and irrelevant each have success 1.0000, while full versus refuting has success 0.9679 (Qiu et al., 5 May 2026). This supports the claim that the method is sensitive to evidence sufficiency rather than only surface features.

The paper also uses a cross-task comparison with GPT-4o to define the task boundary. On matched HotpotQA-RAG v3 examples, GPT-4o: 0.7284 Macro-F1 and SURE-RAG: 0.8951 Macro-F1 (Qiu et al., 5 May 2026). On HaluBench unsafe detection, the ranking reverses: GPT-4o: 0.7389 unsafe-F1, SURE-RAG: 0.3343 unsafe-F1, and best pooling baseline: 0.4863 (Qiu et al., 5 May 2026). The paper treats this reversal as direct evidence that controlled evidence sufficiency verification and natural hallucination detection are distinct problems.

In the paper’s distinction, sufficiency verification asks whether the candidate answer is justified by the retrieved evidence, and it is a three-way evidence-grounded decision over supported, refuted, and insufficient. Hallucination detection is broader, usually binary, more open-ended, and affected by longer outputs and broader semantic issues (Qiu et al., 5 May 2026). A response may be factually plausible but still unsupported by retrieved evidence; likewise, refuted and insufficient are semantically distinct in sufficiency verification. This suggests that a selective evidence-grounded finalizer is best understood not as a general hallucination detector, but as a specialized verifier for evidence-conditioned answer authorization.