Doubly Ranked Testing Methods

• Doubly Ranked Testing is a design principle that constructs an informative one-dimensional order from complex observations, facilitating classical nonparametric tests in settings where direct ranking is unavailable.
• It employs a two-stage ranking process where an initial scoring function or curve summary is learned and then re-ranked to enable tests like Mann–Whitney or Kruskal–Wallis with exact null calibration.
• The methodology offers nonasymptotic error guarantees and computational efficiency, making it versatile for multivariate, functional, and ranking data even in high-dimensional scenarios.

Searching arXiv for the cited papers to ground the article in the current record. Doubly ranked testing denotes a family of nonparametric inference procedures in which ranking is carried out in two stages in order to extend classical rank-based tests to complex observations. In recent arXiv literature, the idea appears in at least three technically distinct forms: a bipartite-ranking reduction of the multivariate two-sample problem on $\mathbb R^d$ (Clémençon et al., 2023), a functional-data procedure that ranks observations at each measurement occurrence, summarizes the resulting rank trajectories, and then re-ranks those summaries before applying Mann–Whitney–Wilcoxon or Kruskal–Wallis tests (Meyer, 2023), and a broader two-sample framework for ranked preference or pairwise-comparison data in which the sample space is itself ranking-valued and inference is based on a universal quadratic statistic with permutation calibration (Rastogi et al., 2020). Taken together, these works treat doubly ranked testing less as a single statistic than as a design principle for constructing order information where no natural scalar order is available.

1. Terminological scope and common construction

The shared motivation is the classical two-sample problem under data modalities for which direct ranking is nontrivial. In the multivariate setting, the obstacle is the lack of natural order on $\mathbb R^d$ for $d\ge 2$ (Clémençon et al., 2023). In grouped functional data, the obstacle is that the unit of observation is a curve rather than a scalar, so any rank-based test must first define how curves are to be ranked (Meyer, 2023). In ranked-preference data, the observations are already partial rankings, total rankings, or pairwise comparisons, and the inferential problem concerns equality of the underlying ranking distributions (Rastogi et al., 2020).

Setting	First ordering step	Final inferential step
Multivariate two-sample data	Learn a scoring function $\hat f:\mathbb R^d\to\mathbb R$ by bipartite ranking	Apply a univariate linear-rank test to held-out scores
Grouped functional data	Rank pooled values at each time point and summarize each curve’s rank vector	Re-rank the summaries and apply MWW or KW
Ranked preference or pairwise-comparison data	Reduce rankings to comparison counts via rank-breaking when needed	Apply a quadratic U-type statistic with analytic or permutation calibration

This suggests that “doubly ranked” is best understood as an architecture: first manufacture an order or score from structured observations, then apply a rank-based or rank-derived test to that induced univariate representation. The concrete implementation differs sharply across Euclidean, functional, and combinatorial sample spaces.

2. Bipartite-ranking reduction for the multivariate two-sample problem

For independent samples

$$\{X_1,\dots,X_n\}\sim G,\qquad \{Y_1,\dots,Y_m\}\sim H,$$

the method of "A Bipartite Ranking Approach to the Two-Sample Problem" splits each sample into training and test parts of sizes $(n',m')$ and $(n'',m'')$ (Clémençon et al., 2023). On the training half,

$D'=\{X_1,\dots,X_{n'};\,Y_1,\dots,Y_{m'}\},$

the observations are labeled $+1$ for the $X$-sample and $\mathbb R^d$0 for the $\mathbb R^d$1-sample. A scoring function $\mathbb R^d$2 is then learned by minimizing a pairwise convex surrogate to

$\mathbb R^d$3

or equivalently by maximizing the empirical AUC,

$\mathbb R^d$4

A popular convex program is the RankSVM-type objective

$\mathbb R^d$5

with $\mathbb R^d$6.

The paper also formulates the learning stage through a two-sample linear-rank criterion. One solves

$\mathbb R^d$7

where

$\mathbb R^d$8

$\mathbb R^d$9 is a score-generating function, and $d\ge 2$0 is a function class such as an RKHS or linear models. Once $d\ge 2$1 has been fixed, it induces a preorder on $d\ge 2$2 via

$d\ge 2$3

The held-out half

$d\ge 2$4

is thereby projected to a univariate sample of scores $d\ge 2$5. The original multivariate hypothesis $d\ge 2$6 versus $d\ge 2$7 is turned into testing whether the univariate distributions of $d\ge 2$8 and $d\ge 2$9 coincide. The central claim is that the learned score projects the data onto the real line nearly like any monotone transform of the likelihood ratio between the original multivariate distributions would do, ignoring ranking model bias issues, and thus preserves the advantages of univariate rank tests while attenuating direct dependence on ambient dimension (Clémençon et al., 2023).

3. Test statistic, calibration, and nonasymptotic guarantees

On the held-out scores, the second stage computes the two-sample linear-rank statistic

$\hat f:\mathbb R^d\to\mathbb R$0

For $\hat f:\mathbb R^d\to\mathbb R$1, this is the Mann–Whitney–Wilcoxon statistic; for general nondecreasing $\hat f:\mathbb R^d\to\mathbb R$2, it is a linear-rank statistic (Clémençon et al., 2023). Under $\hat f:\mathbb R^d\to\mathbb R$3, the ranks of the positive scores are uniform on $\hat f:\mathbb R^d\to\mathbb R$4, with $\hat f:\mathbb R^d\to\mathbb R$5, so

$\hat f:\mathbb R^d\to\mathbb R$6

has a known distribution, either tabulated or easily simulated. If $\hat f:\mathbb R^d\to\mathbb R$7 denotes its $\hat f:\mathbb R^d\to\mathbb R$8-quantile, the rejection rule

$\hat f:\mathbb R^d\to\mathbb R$9

guarantees exact level $$\{X_1,\dots,X_n\}\sim G,\qquad \{Y_1,\dots,Y_m\}\sim H,$$0.

The nonasymptotic analysis separates concentration of the rank statistic from learning error. Under minimal smoothness assumptions on $$\{X_1,\dots,X_n\}\sim G,\qquad \{Y_1,\dots,Y_m\}\sim H,$$1—nondecreasing, bounded, and $$\{X_1,\dots,X_n\}\sim G,\qquad \{Y_1,\dots,Y_m\}\sim H,$$2—the null tail obeys

$$\{X_1,\dots,X_n\}\sim G,\qquad \{Y_1,\dots,Y_m\}\sim H,$$3

with $$\{X_1,\dots,X_n\}\sim G,\qquad \{Y_1,\dots,Y_m\}\sim H,$$4, and hence

$$\{X_1,\dots,X_n\}\sim G,\qquad \{Y_1,\dots,Y_m\}\sim H,$$5

On the training half, if $$\{X_1,\dots,X_n\}\sim G,\qquad \{Y_1,\dots,Y_m\}\sim H,$$6 has finite VC-dimension and $$\{X_1,\dots,X_n\}\sim G,\qquad \{Y_1,\dots,Y_m\}\sim H,$$7 satisfies the Sobolev-$$\{X_1,\dots,X_n\}\sim G,\qquad \{Y_1,\dots,Y_m\}\sim H,$$8 condition, then

$$\{X_1,\dots,X_n\}\sim G,\qquad \{Y_1,\dots,Y_m\}\sim H,$$9

where $(n',m')$0 (Clémençon et al., 2023).

The resulting error decomposition is

$(n',m')$1

The first bracket is univariate noise, the second is bipartite-ranking error under model-bias control, and the third is the signal $(n',m')$2. For any $(n',m')$3 and $(n',m')$4 large enough,

$(n',m')$5

The high-dimensional significance of this construction is explicit. The second stage has exponential-in-$(n',m')$6 error bounds that do not deteriorate as ambient dimension $(n',m')$7 grows because it depends only on one-dimensional ranks of $(n',m')$8. The first stage is adaptive: it learns $(n',m')$9 to mimic an unknown increasing transform of the likelihood ratio $(n'',m'')$0. By varying $(n'',m'')$1, one can emphasize different parts of the ROC curve, including top-end “push” tests for small-mass departures. The paper contrasts this with plug-in density-based tests and RKHS-MMD methods, whose calibration and bandwidth choices may degrade with $(n'',m'')$2 (Clémençon et al., 2023).

4. Doubly ranked tests of location for grouped functional data

Meyer’s functional-data formulation begins from two independent samples of curves,

$(n'',m'')$3

measured on a common discrete grid $(n'',m'')$4 (Meyer, 2023). The null is the pointwise stochastic-equality hypothesis

$(n'',m'')$5

versus inequality for some $(n'',m'')$6. Under a location-shift model $(n'',m'')$7, this becomes $(n'',m'')$8 versus $(n'',m'')$9.

The method has three stages. In Stage 1, at each $D'=\{X_1,\dots,X_{n'};\,Y_1,\dots,Y_{m'}\},$0, the pooled values

$D'=\{X_1,\dots,X_{n'};\,Y_1,\dots,Y_{m'}\},$1

are assigned combined ranks

$D'=\{X_1,\dots,X_{n'};\,Y_1,\dots,Y_{m'}\},$2

with ties broken by the usual mid-rank rule. Under $D'=\{X_1,\dots,X_{n'};\,Y_1,\dots,Y_{m'}\},$3, each $D'=\{X_1,\dots,X_{n'};\,Y_1,\dots,Y_{m'}\},$4 exchangeably. In Stage 2, each subject’s rank trajectory

$D'=\{X_1,\dots,X_{n'};\,Y_1,\dots,Y_{m'}\},$5

is reduced to a scalar summary in one of two ways. The sufficient-statistic summary uses

$D'=\{X_1,\dots,X_{n'};\,Y_1,\dots,Y_{m'}\},$6

Under $D'=\{X_1,\dots,X_{n'};\,Y_1,\dots,Y_{m'}\},$7, $D'=\{X_1,\dots,X_{n'};\,Y_1,\dots,Y_{m'}\},$8 is the natural sufficient statistic in an exponential-family approximation of the $D'=\{X_1,\dots,X_{n'};\,Y_1,\dots,Y_{m'}\},$9th order-statistic distribution, and $+1$0. The alternative summary is the average rank

$+1$1

for which $+1$2 and $+1$3 as $+1$4. Stage 3 re-ranks the pooled summaries:

$+1$5

or analogously for $+1$6.

The two-sample test statistic is then

$+1$7

equivalently

$+1$8

exactly in the form of the Wilcoxon–Mann–Whitney statistic. Under $+1$9, the null distribution is computed by standard permutation enumerations for $X$0 and no ties; with ties or large $X$1, the asymptotic approximation is

$X$2

The framework extends to $X$3 groups. After the same ranking and summarization stages, the final ranks $X$4 yield the doubly ranked Kruskal–Wallis statistic

$X$5

with exact small-$X$6 tables and asymptotic $X$7. An implementation-oriented algorithm also permits optional preprocessing of raw curves by functional-PCA or smoothing, such as FACE, before the ranking stages (Meyer, 2023).

5. Ranked preference and pairwise-comparison data

A broader usage of doubly ranked testing in the supplied literature concerns two-sample testing when each sample consists of ranked preference data or pairwise comparisons rather than Euclidean vectors or functional trajectories (Rastogi et al., 2020). With $X$8 items and two independent batches, the pairwise-comparison model is governed by two unknown win-probability matrices

$X$9

with no ties and $\mathbb R^d$00. The hypothesis is

$\mathbb R^d$01

For full or partial rankings, one instead tests equality of two distributions $\mathbb R^d$02 and $\mathbb R^d$03 over permutations or partial rankings, often through their induced pairwise-comparison matrices.

The paper introduces a minimax risk

$\mathbb R^d$04

and studies the critical separation $\mathbb R^d$05 at which $\mathbb R^d$06. The central statistic is a quadratic U-type statistic built from comparison counts $\mathbb R^d$07, where $\mathbb R^d$08 and $\mathbb R^d$09 are the numbers of observed $\mathbb R^d$10 outcomes in the two batches. In a fixed-design setting with exactly $\mathbb R^d$11 comparisons per pair in each batch, the statistic is computable in $\mathbb R^d$12 time. Calibration can be analytic, using expectation and variance bounds plus Chebyshev, or permutation-based, which controls Type I error exactly at level $\mathbb R^d$13 (Rastogi et al., 2020).

The finite-sample and minimax results are sharp. In the per-pair fixed-design setting, if $\mathbb R^d$14 and

$\mathbb R^d$15

then the test has sum of Type I plus Type II errors at most $\mathbb R^d$16, equivalently

$\mathbb R^d$17

In the random-design setting, if $\mathbb R^d$18 is the mean number of observations per edge and

$\mathbb R^d$19

the same algorithm succeeds with error at most $\mathbb R^d$20; for $\mathbb R^d$21 this becomes

$\mathbb R^d$22

The role of modeling assumptions is a major theme. For model-free, WST, and MST classes, the lower bound remains

$\mathbb R^d$23

so weak or moderate stochastic transitivity does not improve the minimax rate. For SST and parameter-based models such as BTL and Thurstone, the information-theoretic separation is

$\mathbb R^d$24

creating a polynomial gap relative to the model-free setting. Under the planted-clique conjecture, there is also a computational lower bound for SST: in the $\mathbb R^d$25 regime, no polynomial-time test can improve on

$\mathbb R^d$26

For partial and full rankings, the methodology uses rank-breaking—random disjoint, deterministic disjoint, or complete—to extract pairwise comparisons, after which the same statistic is applied. Theorems for partial rankings give

$\mathbb R^d$27

while for full rankings $\mathbb R^d$28 a deterministic disjoint scheme yields

$\mathbb R^d$29

6. Empirical behavior, applications, and interpretive issues

The empirical record in grouped functional data is unusually explicit. Meyer reports simulations under Gaussian and heavy-tailed ($\mathbb R^d$30) bases, with shift patterns

$\mathbb R^d$31

and noise models consisting of none, white, and AR(1) noise (Meyer, 2023). At nominal $\mathbb R^d$32, Type I error for both the sufficient-statistic and average-rank summaries is nearly exactly $\mathbb R^d$33 across $\mathbb R^d$34 and grid sizes $\mathbb R^d$35. Power uniformly dominates a depth-based Mann–Whitney functional test of López-Pintado, Sun, and Genton for all $\mathbb R^d$36, all sample sizes, and all shift shapes, while the two summaries have virtually identical power.

The same paper provides three case studies. In resin viscosity data, involving 64 curves of viscosity over 838 s under 5 binary factors, the doubly ranked MWW test using the sufficient-statistic summary found a resin-temperature effect with $\mathbb R^d$37, $\mathbb R^d$38, a tool-temperature effect with $\mathbb R^d$39, $\mathbb R^d$40, and no significance for the other factors. In Canadian weather data, using 35 stations’ daily-mean temperature and precipitation from 1960–1994 across Arctic, Atlantic, Continental, and Pacific regions, the doubly ranked KW test yielded $\mathbb R^d$41, $\mathbb R^d$42 for temperature and $\mathbb R^d$43, $\mathbb R^d$44 for precipitation. In COVID-19 mobility data, county-level daily driving-request-change curves grouped by state produced $\mathbb R^d$45, $\mathbb R^d$46 for Colorado versus Utah, $\mathbb R^d$47, $\mathbb R^d$48 for Iowa versus Minnesota, and $\mathbb R^d$49, $\mathbb R^d$50 for Maryland versus Virginia versus West Virginia (Meyer, 2023).

For ranked-preference data, simulations under symmetric, model-free, random-design $\mathbb R^d$51 observations show empirical power curves collapsing as predicted by the rate $\mathbb R^d$52 (Rastogi et al., 2020). Real-world datasets reinforce the practical scope: in crowdsourcing data from six AMT tasks, the permutation test gave $\mathbb R^d$53, rejecting equality between direct pairwise comparisons and comparisons obtained by thresholding ratings; in European football across two seasons, the result was $\mathbb R^d$54, failing to reject a shift in relative team-strength distributions; and in Sushi preference data, demographic splits by gender, age, and region yielded $\mathbb R^d$55, indicating distinct preference distributions.

Several interpretive issues recur across these literatures. First, doubly ranked testing is not identical to depth-based scoring. In the functional setting, several procedures based on depth scores are criticized because the scores are not constructed under the null and often introduce additional, uncontrolled for variability (Meyer, 2023). Second, the phrase does not denote one canonical formula. In one line of work it means a learned preorder followed by a univariate rank test; in another it means pointwise ranking followed by re-ranking of curve summaries; in a broader ranking-data context it refers to inference on ranking-valued observations themselves. This suggests that the defining feature is a two-stage exploitation of ordinal information rather than a unique statistic. Third, the effect of structural assumptions is subtle: strong modeling assumptions such as SST or parametric links can improve information-theoretic separation in ranking-data testing, but may not yield comparable computational gains (Rastogi et al., 2020). Across the cited works, the most stable advantages are exact or near-exact null calibration, robustness inherited from rank procedures, and the ability to recover power on high-dimensional, functional, or combinatorial domains by constructing an informative one-dimensional order before testing.