Conformal Set Ranked Retrieval (CSR)

• CSR is a set-valued uncertainty quantification technique that replaces a single ranked output with a small, guaranteed superset containing the true target.
• It employs geometric and sampling-based methods—such as Stripe Range Retrieval and the EpsRange algorithm—to construct narrow score bands with controlled set sizes.
• Extensions include online calibration, two-stage risk control, and order-sensitive constructions that enhance robustness in retrieval and ranking tasks.

Searching arXiv for the specified papers and closely related CSR work. arxiv_search(query="(Ge et al., 2024) OR (Intrator et al., 2024) OR (Xu et al., 2024) OR (Dehghankar et al., 1 Aug 2025) OR (Liao et al., 30 Jan 2026) OR (Luo et al., 2024) OR (Fermanian et al., 20 Jan 2025) OR (Haas et al., 23 Jun 2026) OR (Alkhatib et al., 27 Mar 2026) OR (Jia et al., 22 Nov 2025)", max_results=10) Conformal Set Ranked Retrieval (CSR) denotes set-valued uncertainty quantification for ranked access and ranked retrieval. In its most explicit formulation, CSR is introduced as a relaxed version of Direct-Access Ranked Retrieval (DAR): instead of returning the exact tuple at target rank $i$, it returns a small subset guaranteed to contain that tuple (Dehghankar et al., 1 Aug 2025). Closely related work calibrates thresholded retrieval prefixes, candidate sets, or sets of plausible absolute ranks for ranked systems, often with finite-sample marginal guarantees under exchangeability or i.i.d. sampling (Intrator et al., 2024, Ge et al., 2024, Fermanian et al., 20 Jan 2025, Liao et al., 30 Jan 2026). This suggests a broader interpretation of CSR as an umbrella for conformal or conformal-style methods that replace brittle pointwise ranked outputs with calibrated sets over items, cutoffs, or rank positions.

1. Formal definition and semantic scope

In the direct-access formulation, the dataset is static: $D=\{p_1,p_2,\dots,p_n\}\subset \mathbb{R}^d,$ a query provides a linear scoring function $f\in\mathbb{R}^d$, and the score of a point $p$ is

$score_f(p)=f^\top p=\sum_{j=1}^d f[j]\,p[j],$

with $\|f\|=1$. Sorting points by descending score induces a rank map $rank_{D,f}$, and the tuple at rank $i$ is

$p^{(i)}=rank^{-1}_{D,f}(i).$

Exact DAR asks for $p^{(i)}$. CSR relaxes this by asking for a subset $D=\{p_1,p_2,\dots,p_n\}\subset \mathbb{R}^d,$0 such that

$D=\{p_1,p_2,\dots,p_n\}\subset \mathbb{R}^d,$1

for a small $D=\{p_1,p_2,\dots,p_n\}\subset \mathbb{R}^d,$2. The guarantee is containment of the exact rank-$D=\{p_1,p_2,\dots,p_n\}\subset \mathbb{R}^d,$3 tuple, not approximation in score or approximation in rank (Dehghankar et al., 1 Aug 2025).

That formulation also clarifies an important terminological nuance. In (Dehghankar et al., 1 Aug 2025), the phrase “conformal set” is inspired by conformal prediction, but it is not defined through a probability calibration procedure; it is a guaranteed superset of the target tuple with controlled size. In adjacent retrieval and ranking papers, by contrast, “conformal” usually refers to split conformal prediction, conformal risk control, or online conformal calibration. The shared structural idea is that a ranked system does not commit to a single position or a single returned object; it returns a calibrated set whose size is part of the design problem.

Geometrically, CSR in the direct-access setting is a projection problem. A scoring function $D=\{p_1,p_2,\dots,p_n\}\subset \mathbb{R}^d,$4 orders tuples by $D=\{p_1,p_2,\dots,p_n\}\subset \mathbb{R}^d,$5, so target rank corresponds to order along direction $D=\{p_1,p_2,\dots,p_n\}\subset \mathbb{R}^d,$6. The relaxed problem replaces exact identification of a single arrangement level by identifying a narrow score band that necessarily contains the target tuple (Dehghankar et al., 1 Aug 2025).

2. Geometric and sampling-based CSR algorithms

The main algorithmic route in (Dehghankar et al., 1 Aug 2025) is to reduce CSR to Stripe Range Retrieval (SRR). A stripe range is

$D=\{p_1,p_2,\dots,p_n\}\subset \mathbb{R}^d,$7

and SRR asks for $D=\{p_1,p_2,\dots,p_n\}\subset \mathbb{R}^d,$8. If one can choose $D=\{p_1,p_2,\dots,p_n\}\subset \mathbb{R}^d,$9 and $f\in\mathbb{R}^d$0 so that the unknown score of $f\in\mathbb{R}^d$1 lies between them, then the returned stripe intersection is a valid CSR set.

The paper constructs these score bounds using an $f\in\mathbb{R}^d$2-sample $f\in\mathbb{R}^d$3. If $f\in\mathbb{R}^d$4, the sample ranks

$f\in\mathbb{R}^d$5

identify sample points $f\in\mathbb{R}^d$6 and $f\in\mathbb{R}^d$7, which induce thresholds

$f\in\mathbb{R}^d$8

The boundary lemma gives

$f\in\mathbb{R}^d$9

so

$p$0

The resulting CSR output has size $p$1. For stripe ranges, the required $p$2-sample size is

$p$3

up to the failure-probability term. The EpsRange algorithm solves CSR with linear space and query time

$p$4

and the paper characterizes this as near-optimal up to logarithmic factors under the lower-bound connection to half-space range counting (Dehghankar et al., 1 Aug 2025).

For high dimensions, (Dehghankar et al., 1 Aug 2025) also introduces EpsHier, which accelerates the SRR subroutine using hierarchical sampling. The hierarchy recursively subsamples layers $p$5, assigns each point to a nearest centroid in the next layer, and stores enclosing balls for the represented base-layer points. At query time, nodes whose enclosing balls do not intersect the stripe are pruned. This variant retains linear space, has preprocessing time $p$6, and is reported as practically efficient even though the worst-case query bound in Table 1 remains $p$7. Empirically, hierarchical sampling achieves up to $p$8 speedup over exhaustive search for narrow stripe queries, and CSR outputs have $p$9 recall in all experiments (Dehghankar et al., 1 Aug 2025).

3. Thresholded prefixes and online conformal retrieval

A second line of work interprets CSR through score-thresholded retrieval sets. In the online semi-bandit setting of “Stochastic Online Conformal Prediction with Semi-Bandit Feedback” (Ge et al., 2024), each round receives $score_f(p)=f^\top p=\sum_{j=1}^d f[j]\,p[j],$0, predicts

$score_f(p)=f^\top p=\sum_{j=1}^d f[j]\,p[j],$1

and observes the true label only if it lies in the prediction set. In the retrieval application, $score_f(p)=f^\top p=\sum_{j=1}^d f[j]\,p[j],$2 is a document scorer; with Dense Passage Retriever on SQuAD, the score is cosine similarity between question and document embeddings. When scores induce a ranking, thresholding produces a prefix whenever there are no score ties or ties are broken consistently. The algorithm, Semi-bandit Prediction Set (SPS), estimates the CDF of true-label scores conservatively via truncation and a DKW confidence band, updates thresholds monotonically upward, achieves regret

$score_f(p)=f^\top p=\sum_{j=1}^d f[j]\,p[j],$3

and with probability at least $score_f(p)=f^\top p=\sum_{j=1}^d f[j]\,p[j],$4 maintains $score_f(p)=f^\top p=\sum_{j=1}^d f[j]\,p[j],$5, hence $score_f(p)=f^\top p=\sum_{j=1}^d f[j]\,p[j],$6. On SQuAD retrieval, SPS is reported to achieve the lowest cumulative regret, zero undercoverage count in practice, and a better coverage-efficiency tradeoff than the baselines under retrieval-style partial feedback (Ge et al., 2024).

“Streamlining Conformal Information Retrieval via Score Refinement” inserts a score-shaping layer into a standard conformal retrieval pipeline (Intrator et al., 2024). Documents receive raw similarity scores

$score_f(p)=f^\top p=\sum_{j=1}^d f[j]\,p[j],$7

and vanilla conformal retrieval uses $score_f(p)=f^\top p=\sum_{j=1}^d f[j]\,p[j],$8 with

$score_f(p)=f^\top p=\sum_{j=1}^d f[j]\,p[j],$9

The refinement normalizes each query by $\|f\|=1$0 and applies a rank discount based on $\|f\|=1$1, with $\|f\|=1$2. The paper states that the transformation is monotone and preserves the IR model’s induced order, so the final conformal retrieval set is a thresholded prefix of the refined ranking. Under exchangeability, the targeted guarantee remains the standard split-conformal statement

$\|f\|=1$3

On FEVER, FIQA, and SCIFACT, the method usually yields much smaller average set sizes than baseline CP, though the paper also reports empirical undercoverage on some small datasets and does not provide a theorem proving smaller sets without sacrificing validity (Intrator et al., 2024).

4. Two-stage retrieval–ranking calibration

CSR also appears in two-stage systems where retrieval and ranking are calibrated jointly. “Two-stage Risk Control with Application to Ranked Retrieval” models a sequential pipeline in which retrieval produces

$\|f\|=1$4

and ranking produces

$\|f\|=1$5

The retrieval-stage loss is recall/FNR-style,

$\|f\|=1$6

while the ranking-stage loss is

$\|f\|=1$7

Thresholds are chosen by the conformal risk control inequalities

$\|f\|=1$8

or by a joint optimization over $\|f\|=1$9 that minimizes combined set size subject to the risk constraints (Xu et al., 2024).

The guarantee is not standard label coverage. It is finite-sample, distribution-free marginal expected-risk control under exchangeability: $rank_{D,f}$0 This makes the framework closer to conformal risk control than to split conformal classification, but it is directly relevant to CSR because it calibrates candidate-set size and ranked-list size inside a retrieval pipeline. Experiments on MSLR-WEB, Yahoo LTRC, and MS MARCO show that actual retrieval risk tracks the target $rank_{D,f}$1, actual ranking risk tracks the target $rank_{D,f}$2, and combined prediction sizes are minimized near the maximal feasible retrieval risk for the ranking objective (Xu et al., 2024).

5. Rank-position uncertainty and order-sensitive set construction

Another major CSR theme is uncertainty over positions rather than only over membership. “Transductive Conformal Inference for Full Ranking” constructs prediction sets $rank_{D,f}$3 for the absolute ranks of new items when only the relative ranking of $rank_{D,f}$4 calibration items is known (Fermanian et al., 20 Jan 2025). Because the full ranks of calibration items are unobserved, the method builds simultaneous envelopes $rank_{D,f}$5 for those latent ranks, defines proxy calibration scores

$rank_{D,f}$6

and then returns

$rank_{D,f}$7

For the residual-rank score, the output is a symmetric interval around the predicted rank; for the value-adaptive score, interval width depends on local score spacing. The method provides marginal coverage for each test-item rank and high-probability false coverage proportion control across multiple test items (Fermanian et al., 20 Jan 2025).

“Distribution-informed Efficient Conformal Prediction for Full Ranking” refines this direction by deriving the exact conditional distribution of latent calibration absolute ranks (Liao et al., 30 Jan 2026). Conditional on a calibration item’s relative rank $rank_{D,f}$8, the increment $rank_{D,f}$9 follows a Negative Hypergeometric distribution, which yields an exact score distribution and a mixture CDF $i$0 for latent calibration scores. DCR chooses the threshold from $i$1 rather than from worst-case score envelopes, preserves valid marginal coverage for rank sets, and satisfies

$i$2

hence smaller prediction sets than TCPR. The paper reports up to $i$3 reduction in average prediction set size while maintaining valid coverage (Liao et al., 30 Jan 2026).

Several adjacent papers contribute order-sensitive design principles that are not retrieval methods in the IR sense but are directly relevant to CSR. “Reliable Conformal Prediction for Ordinal Classification Using the Ranked Probability Score” uses the Ranked Probability Score as nonconformity and proves that the resulting conformal sets are contiguous, nested, and median-centered by construction (Haas et al., 23 Jun 2026). This suggests a CSR formulation over ordered target variables such as relevant-item rank position, top-$i$4 cutoff depth, or ordinal relevance grade. “Trustworthy Classification through Rank-Based Conformal Prediction Sets” calibrates the rank of the true label and returns either the top-$i$5 or top-$i$6 labels, with finite-sample marginal coverage under exchangeability (Luo et al., 2024). “Cost-Sensitive Conformal Training with Provably Controllable Learning Bounds” proves that expected prediction set size is upper bounded by expected true-label rank plus calibration slack and proposes Rank Weighted Cross-Entropy, reducing average prediction set size by $i$7 in the reported experiments (Jia et al., 22 Nov 2025). “Contrastive Conformal Sets” moves the set construction into embedding space, learning minimum-volume conformal neighborhoods around anchor embeddings with guaranteed positive coverage; it is a strong precursor for set retrieval in semantic spaces, although it does not provide a ranked retrieval guarantee (Alkhatib et al., 27 Mar 2026).

6. Assumptions, limitations, and open directions

The CSR literature is technically heterogeneous, and its guarantees are correspondingly heterogeneous. In (Dehghankar et al., 1 Aug 2025), CSR assumes a static dataset and linear scoring functions $i$8; dynamic updates and non-linear ranking functions are listed as future directions. The direct-access guarantee is deterministic containment, not probabilistic conformal calibration, and the output size scales as $i$9, so very small returned sets require small $p^{(i)}=rank^{-1}_{D,f}(i).$0 and therefore larger samples.

In conformal retrieval papers, assumptions are typically i.i.d. sampling or exchangeability. The online semi-bandit framework in (Ge et al., 2024) assumes i.i.d. samples from a fixed distribution, a single true label per query, and threshold-defined sets; it does not optimize precision@$p^{(i)}=rank^{-1}_{D,f}(i).$1, MRR, NDCG, or any ordering-sensitive retrieval utility. The score-refinement pipeline in (Intrator et al., 2024) relies on exchangeability, calibrates only after top-2000 ANN truncation, reduces multi-relevant-document tasks to the highest-scoring relevant document, and provides no formal efficiency theorem. The two-stage CRC framework in (Xu et al., 2024) gives marginal expected-risk guarantees, not per-query or conditional guarantees, and its losses are specific to recall/FNR-style retrieval coverage and nDCG-style ranking quality.

Full-ranking conformal methods likewise have restrictive assumptions. TCPR and DCR both assume exchangeability of a finite population of items and no ties in latent scores; the output is a set of plausible absolute ranks, not a conformal set of documents or a direct top-$p^{(i)}=rank^{-1}_{D,f}(i).$2 retrieval guarantee (Fermanian et al., 20 Jan 2025, Liao et al., 30 Jan 2026). Adjacent order-sensitive papers are strongest when the output space is genuinely one-dimensional and ordered, and their transfer to retrieval becomes less direct when there are multiple relevant documents, incomplete judgments, diversity constraints, or noncontiguous desired outputs (Haas et al., 23 Jun 2026, Luo et al., 2024).

Open directions stated across the literature are comparatively consistent. They include direct calibration for “at least one relevant among many,” query-conditional or difficulty-adaptive conformal retrieval, learned monotone score warping, integration with rerankers or cascaded retrieval, dynamic datasets, non-linear ranking functions, top-$p^{(i)}=rank^{-1}_{D,f}(i).$3-specific guarantees, and stronger treatment of multiple relevant items and graded relevance (Intrator et al., 2024, Dehghankar et al., 1 Aug 2025, Fermanian et al., 20 Jan 2025). Taken together, these works position CSR not as a single algorithmic template, but as a technically varied research program centered on one question: how to replace brittle ranked point predictions by compact, query-relevant sets that retain formal guarantees over inclusion, risk, or rank position.