Tolerance-Corrected Win-and-Tie Rates

• The paper introduces tolerance-corrected win-and-tie rates by defining wins, losses, and ties with a user-specified tolerance to capture meaningful differences.
• It details IPCW-based estimation methods for right-censored clinical data and extends these concepts to sports analytics using margin-specific Elo ratings.
• Key simulation results show low estimator bias and robust variance properties, validating the approach for both clinical trials and competitive game scenarios.

Tolerance-corrected win-and-tie rates are statistical measures that generalize classic win statistics to settings where “wins,” “losses,” and “ties” are defined relative to an explicit margin or tolerance parameter. By introducing a user-specified tolerance—often called an equivalence margin or tie threshold—one can formally account for outcomes that are not meaningfully different, such as negligible differences in event times in clinical trials, or narrow point spreads in games of skill. This approach has found major applications in the analysis of prioritized time-to-event endpoints under censoring in clinical research (Cui et al., 3 Jun 2025), and in the extension of ranking/rating systems such as Elo to handle margin-of-victory and tie definitions in sports analytics (Moreland et al., 2018).

1. Formal Definition of Tolerance-Corrected Win, Loss, and Tie

In both clinical trial and skill-based gaming contexts, tolerance-corrected statistics are constructed by considering pairwise or contest-specific outcomes in light of a prespecified tolerance.

Time-to-Event (Clinical) Settings

• For two arms (“treatment,” size $N_t$; “control,” size $N_c$) with observed times $X = \min(T, C, \tau)$, and event indicator $\Delta = I(T \wedge \tau \leq C)$, an equivalence margin $\delta \geq 0$ is fixed.
• Pairwise outcomes for all $(i, j)$ between treatment and control are defined as:
  • Win: $W_{ij}(\delta) = 1$ if $X_i > X_j + \delta$ and $\Delta_j = 1$
  • Loss: $L_{ij}(\delta) = 1$ if $N_c$0 and $N_c$1
  • Tie: $N_c$2, equivalently $N_c$3

Skill-Based Competitions (Games)

• For a set of possible victory margins $N_c$4, define “win” for margin $N_c$5 as the event $N_c$6.
• Tolerance $N_c$7 is set by the analyst:
  • Win (under $N_c$8): $N_c$9
  • Tie (under $X = \min(T, C, \tau)$0): $X = \min(T, C, \tau)$1, with $X = \min(T, C, \tau)$2

By varying $X = \min(T, C, \tau)$3 (or $X = \min(T, C, \tau)$4), the methodology incorporates clinically or contextually meaningful definitions of practical equivalence, systematically distinguishing strong effects from trivial or random fluctuations.

2. Estimators and Statistical Inference

IPCW-Adjusted Estimation (Clinical Trials)

For right-censored data, inverse-probability-of-censoring weighting (IPCW) is used to correct estimators so that they remain valid in the presence of incomplete follow-up (Cui et al., 3 Jun 2025):

• Estimate the survival function of censoring $X = \min(T, C, \tau)$5 via the pooled Kaplan–Meier estimator $X = \min(T, C, \tau)$6.
• Treatment win-rate:

$X = \min(T, C, \tau)$7

• Control win-rate and tie-rate are analogously constructed; symmetrized IPCW estimators are available for ties.

Margin-of-Victory Extension (Elo-Rating Framework)

For games or matches, tolerance-corrected probabilities are inferred from margin-specific Elo ratings (Moreland et al., 2018):

• For each team and margin $X = \min(T, C, \tau)$8, maintain $X = \min(T, C, \tau)$9 and $\Delta = I(T \wedge \tau \leq C)$0.
• After each match, update all relevant margin-specific ratings using observed versus expected outcome at that margin.
• The predicted win probability for tolerance $\Delta = I(T \wedge \tau \leq C)$1 is $\Delta = I(T \wedge \tau \leq C)$2, where $\Delta = I(T \wedge \tau \leq C)$3 is the normal CDF.

3. Large-Sample Properties and Variance Estimation

In the IPCW context (Cui et al., 3 Jun 2025), estimated win and loss rates are two-sample $\Delta = I(T \wedge \tau \leq C)$4-statistics with IPCW contributions:

• Under standard regularity, asymptotic normality holds:

$\Delta = I(T \wedge \tau \leq C)$5

• $\Delta = I(T \wedge \tau \leq C)$6 can be estimated using leave-one-out sums over all pairs.
• For functionals such as win-ratio ($\Delta = I(T \wedge \tau \leq C)$7) or net-benefit ($\Delta = I(T \wedge \tau \leq C)$8), the delta method provides variance formulas; CIs are derived using normal approximations (often on the log scale for win-ratio).

4. Choice of Tolerance Parameter and Sensitivity Analysis

The equivalence margin $\Delta = I(T \wedge \tau \leq C)$9 (or tolerance $\delta \geq 0$0) is crucial and determined a priori based on clinical, statistical, or game-theoretic relevance:

• In clinical settings, $\delta \geq 0$1 reflects the smallest clinically important difference (typical values: 2–4 months for overall survival; 1–2 months for progression-free survival).
• The protocol should pre-specify $\delta \geq 0$2 after consensus among stakeholders.
• Sensitivity analysis over a grid of $\delta \geq 0$3 or $\delta \geq 0$4 values is recommended to assess the robustness of inferences; moderate increases typically reduce $\delta \geq 0$5 and $\delta \geq 0$6 symmetrically, leaving ratios (e.g., $\delta \geq 0$7) stable.

5. Key Simulation Results and Empirical Illustration

Extensive simulations in (Cui et al., 3 Jun 2025) support the finite-sample validity of IPCW-adjusted tolerance-corrected win statistics under right-censoring:

• Bias for estimators ($\delta \geq 0$8, $\delta \geq 0$9, $(i, j)$0) is $(i, j)$1 even for small $(i, j)$2.
• Analytical standard errors agree with empirical SDs to within $(i, j)$3; 95% CI coverage is near nominal (94–96%).
• Win-ratio-based tests (with $(i, j)$4) show substantially higher power (e.g., $(i, j)$5) compared to log-rank tests in certain settings.
• Moderate increases in $(i, j)$6 reduce both $(i, j)$7 and $(i, j)$8 but leave $(i, j)$9 nearly unchanged.

A real-data application (JAVELIN Renal 101, N=886) illustrates the approach:

$W_{ij}(\delta) = 1$0 (months)	$W_{ij}(\delta) = 1$1	$W_{ij}(\delta) = 1$2	$W_{ij}(\delta) = 1$3	95% CI	$W_{ij}(\delta) = 1$4-value
0	0.57	0.34	1.66	[1.24, 2.22]	0.0004
2	–	–	1.65	[1.28, 2.12]	<0.0001
4	–	–	1.72	[1.35, 2.19]	<0.0001

The stability of $W_{ij}(\delta) = 1$5 across $W_{ij}(\delta) = 1$6 demonstrates robustness and interpretability of tolerance-corrected metrics.

6. Algorithmic and Applied Extensions

The tolerance-corrected framework generalizes to multiple prioritized endpoints (as in win statistics with ordered outcomes), margin-of-victory contests, and settings with arbitrary censoring.

• IPCW-based approaches are nonparametric and avoid hazard modeling, yielding interpretable confidence intervals and tests in finite samples (Cui et al., 3 Jun 2025).
• In sports analytics, constructing the full CDF of point differentials via margin-specific ratings enables computation of win/tie rates for arbitrary tolerances, greatly extending the classical Elo paradigm (Moreland et al., 2018).

A plausible implication is that tolerance-corrected win-and-tie rates provide a unifying formalism for evidence synthesis in settings with ambiguity around “meaningful” effect size or outcome superiority; this is especially valuable when the classic win/loss dichotomy is not clinically or substantively justified.

7. Practical and Conceptual Considerations

Tolerance-corrected methods handle incomplete data, practical equivalence, and multi-endpoint or complex game scenarios in a consistent inferential framework.

Key features include:

• Censoring-robust estimation via IPCW.
• Explicit, pre-specified handling of equivalence through the tolerance parameter.
• Standardized large-sample theory supporting conventional Wald-type inference.
• Generalization to any context where outcome superiority may be ambiguous or context-dependent.

In sum, tolerance-corrected win-and-tie rates enable refined, transparent, and robust quantification of comparative effectiveness or likelihood, advancing both clinical trial methodology and rating system design in games of skill (Cui et al., 3 Jun 2025, Moreland et al., 2018).