Run-Wise Early-Stopping Certificates

• The paper demonstrates that run-wise early-stopping certificates justify halting iterative procedures when the current predictor statistically aligns with the risk minimizer.
• It introduces ScoreStop, a method using functional score tests and chi-square calibrated statistics to decide if further training is operationally redundant.
• The approach is applicable across various tasks—including ranking and survival analysis—offering a principled alternative to traditional patience-based stopping rules.

Run-wise early-stopping certificates are per-execution stopping justifications attached to a concrete training, inference, or search trajectory. Rather than terminating because a fixed iteration cap or patience window has been reached, they halt when an explicit criterion certifies that further computation is either statistically unsupported, operationally unnecessary, or provably unable to improve the incumbent outcome. In gradient boosted decision trees, "ScoreStop: Gradient-based early stopping using functional score tests" formalizes this idea by testing whether the current predictor is the population risk minimizer and issuing a certificate when the estimated directional derivative is statistically indistinguishable from zero (Hines et al., 1 Jun 2026). Related work instantiates the same broad idea with e-processes, conformal thresholds, frontier invariants, run-length statistics, and anytime-valid confidence sequences, but the certified object differs substantially across domains.

1. Definition and scope

In the most direct statistical formulation, a run-wise early-stopping certificate is a decision that the current state of an iterative procedure is consistent with a formally defined target. For ScoreStop, that target is the population risk minimizer $f_0$, and the certificate states that the current predictor $f_t$ is consistent with being the population minimizer when the null hypothesis $H_0: f_t=f_0$ is not rejected on validation data (Hines et al., 1 Jun 2026). This replaces the standard patience rule, which halts after $P$ consecutive non-improvements in validation loss, but whose parameter has no calibrated statistical meaning and behaves idiosyncratically across datasets, sample sizes, losses, learning rates, and tree complexities.

The literature shows that the phrase is broader than a single testing framework. Sequential-EDFL for language-model generation realizes a run-wise certificate through an anytime-valid e-process whose threshold crossing $E_t \ge 1/\delta$ triggers stopping with type I error controlled at level $\delta$ regardless of stopping time (Akter et al., 7 Oct 2025). Statistical early stopping for reasoning models uses either a renewal-process approximation or a nonparametric maxwise conformal rule, with the latter giving a finite-sample per-run bound on the probability of halting too early on well-posed queries under exchangeability (Xie et al., 15 Feb 2026). Auditable routing certificates in agentic search are deterministic and ledger-verified: halting is sound when every unexpanded leaf is upper-bounded by a frontier key under the same realized randomness (Akhauri, 9 Sep 2025).

A common misconception is that all such certificates are anytime-valid or certify correctness. The available work does not support that generalization. ScoreStop’s $\chi^2_d$ calibration is per iteration rather than uniform over sequential stopping times (Hines et al., 1 Jun 2026). Sequential-EDFL certifies information sufficiency relative to a skeleton baseline, not factual correctness (Akter et al., 7 Oct 2025). Reasoning-model certificates bound false-positive early halts on well-posed queries rather than truth of the final answer (Xie et al., 15 Feb 2026). This suggests that “run-wise early-stopping certificate” is best understood as a family of stopping semantics, not a single guarantee.

2. ScoreStop and the functional score-test formulation

ScoreStop is defined for iterative learners that produce predictors $f_1,f_2,\ldots,f_m$, with a focus on gradient boosted decision trees. The construction applies to smooth losses, nonsmooth or implicit losses such as LambdaRank, and data-dependent losses such as Cox regression, where “gradients” may be subgradients or influence-function-based (Hines et al., 1 Jun 2026). The underlying population problem is specified by letting $W=(Y,X)$, $f:X\to\mathbb{R}$ in a Hilbert space $f_t$0, and unit loss $f_t$1, with

$f_t$2

At iteration $f_t$3, the learner holds the current predictor $f_t$4, and the next update direction $f_t$5 is fitted to approximate the gradient function

$f_t$6

ScoreStop then tests the null hypothesis $f_t$7, where $f_t$8 is typically $f_t$9. The relevant local quantity is the Gateaux derivative

$H_0: f_t=f_0$0

with score function

$H_0: f_t=f_0$1

Under $H_0: f_t=f_0$2 and fixed $H_0: f_t=f_0$3, the validation average satisfies

$H_0: f_t=f_0$4

The operational interpretation is that if the directional derivative along the learned update direction is statistically indistinguishable from zero, then the current model is consistent with being the population minimizer and training can stop. Because the construction uses gradients rather than loss values, it avoids the dependence of patience-based rules on noisy or implicitly defined validation losses.

A central design property is scale-invariance. Replacing $H_0: f_t=f_0$5 or $H_0: f_t=f_0$6 for any $H_0: f_t=f_0$7 leaves the test statistic unchanged. The paper states that learning-rate choices and sign conventions therefore do not affect the test, and equivalently

$H_0: f_t=f_0$8

independent of $H_0: f_t=f_0$9 (Hines et al., 1 Jun 2026).

3. Test statistics, stopping rule, and theoretical guarantees

The single-direction ScoreStop statistic uses a plug-in variance estimate: $P$0 Under $P$1, it satisfies

$P$2

For vector-valued predictors $P$3, the score generalizes to

$P$4

For multiple directions $P$5, ScoreStop forms a quadratic statistic

$P$6

with asymptotic null law $P$7 (Hines et al., 1 Jun 2026).

The per-iteration stopping rule is operationally simple. At iteration $P$8, the base learner $P$9 is fit on the training set. On the held-out validation set, one computes $E_t \ge 1/\delta$0. If $E_t \ge 1/\delta$1, training stops and returns $E_t \ge 1/\delta$2; otherwise the update $E_t \ge 1/\delta$3 is accepted and training continues. The threshold is calibrated as

$E_t \ge 1/\delta$4

with $E_t \ge 1/\delta$5 for the forward and backward variants and $E_t \ge 1/\delta$6 for the stabilized two-direction variant. For $E_t \ge 1/\delta$7, the paper notes the convenient parameterization $E_t \ge 1/\delta$8 with two-sided $E_t \ge 1/\delta$9 (Hines et al., 1 Jun 2026).

Three algorithmic variants are distinguished. Forward ScoreStop (FWD-SS) tests $\delta$0. Backward ScoreStop (BWD-SS) tests $\delta$1 for $\delta$2. Stabilized ScoreStop (STAB-SS) uses the two-direction statistic $\delta$3. The computational overhead is one validation pass per iteration to compute $\delta$4 and $\delta$5, or $\delta$6 and $\delta$7 in the data-dependent case; no training refits are needed for the test itself.

The asymptotic theory is organized around assumptions on influence-function centrality, direction linearity and a Riesz representer, finite variance, regular asymptotic linearity, and moment consistency. Under these conditions, the null distribution is $\delta$8 or $\delta$9 as appropriate. The power result states that if $\chi^2_d$0 and the variance is finite, then $\chi^2_d$1 for any fixed $\chi^2_d$2, so the test continues when the current predictor is not the minimizer and the direction is aligned with improvement (Hines et al., 1 Jun 2026). By Cauchy–Schwarz, $\chi^2_d$3 is maximized by $\chi^2_d$4, which the paper identifies with the base learner direction in boosting; this is the sense in which FWD-SS tests in a near-optimal direction.

The main theoretical caveat is sequential. The paper explicitly recommends choosing $\chi^2_d$5 as a performance-tuned regularization parameter, interpretable via $\chi^2_d$6, rather than treating it as a strict test level over the entire stopping time. There are no anytime-valid guarantees for the online use of one test per iteration (Hines et al., 1 Jun 2026).

4. Implicit losses, data-dependent losses, and influence-function generalization

A notable feature of ScoreStop is that it does not depend on explicit evaluation of a scalar loss value. This is crucial for LambdaRank, where the training signal is an implicit document-wise gradient, and for Cox proportional hazards, where the loss depends on nuisance quantities estimated from the data (Hines et al., 1 Jun 2026).

For data-dependent losses, the score $\chi^2_d$7 is replaced by a mean-zero influence function $\chi^2_d$8 satisfying

$\chi^2_d$9

and the corresponding one-step debiased statistic is

$f_1,f_2,\ldots,f_m$0

This reduces to the basic statistic when the score is known and there is no data dependence.

In LambdaRank, a query $f_1,f_2,\ldots,f_m$1 with scores $f_1,f_2,\ldots,f_m$2 uses

$f_1,f_2,\ldots,f_m$3

where the pairwise gradient is

$f_1,f_2,\ldots,f_m$4

and

$f_1,f_2,\ldots,f_m$5

The paper states that Poincaré symmetry ensures an implicit loss exists locally, but ScoreStop uses $f_1,f_2,\ldots,f_m$6 directly and thereby avoids explicit evaluation of nonsmooth NDCG.

For Cox proportional hazards, the unit loss and gradient are

$f_1,f_2,\ldots,f_m$7

On the validation set, the Breslow baseline-hazard estimator is

$f_1,f_2,\ldots,f_m$8

and the efficient influence function under $f_1,f_2,\ldots,f_m$9 is

$W=(Y,X)$0

with

$W=(Y,X)$1

Plugging this $W=(Y,X)$2 into the one-step statistic yields a nuisance-adjusted ScoreStop test. The broader implication is that run-wise certificates need not be restricted to explicit, smooth, scalar losses; they can be built from derivative representations that remain valid under ranking and survival objectives.

5. Empirical behavior, limitations, and operational guidance

In synthetic experiments across regression, classification, ranking, and survival tasks with known $W=(Y,X)$3, ScoreStop in its forward, backward, and stabilized forms is competitive with patience-based stopping at small $W=(Y,X)$4, and is notably stronger in learning-to-rank, where loss-based stopping tracks a nonsmooth metric not directly tied to training gradients (Hines et al., 1 Jun 2026). STAB-SS often outperforms FWD-SS and BWD-SS on regression and survival, which the paper attributes to reduced reliance on a single base learner. Near the test-loss minimum iteration, $W=(Y,X)$5 aligns well with $W=(Y,X)$6 in QQ plots, indicating good finite-sample calibration of the null distribution. At moderate learning rates such as $W=(Y,X)$7, ScoreStop and patience are comparable; very large $W=(Y,X)$8 degrades all methods. Across 13 real datasets spanning regression, classification, multiclass, Poisson, quantile, ranking, and survival, ScoreStop with $W=(Y,X)$9 is broadly competitive with patience baselines, especially when validation losses are noisy or implicit.

The same experiments motivate practical threshold choices. The paper recommends $f:X\to\mathbb{R}$0, corresponding to $f:X\to\mathbb{R}$1 for the single-direction case. It also recommends communicating significance through $f:X\to\mathbb{R}$2, interpreted in standard deviation units, while treating $f:X\to\mathbb{R}$3 as a regularization parameter calibrated to $f:X\to\mathbb{R}$4 rather than as a strict familywise sequential error level.

Several failure modes are explicit. Directional power depends on alignment: if $f:X\to\mathbb{R}$5, the test has no power against that alternative. STAB-SS mitigates this by testing multiple directions, but successive update directions can be nearly collinear, leading to rank-deficient covariance, especially at very small learning rates. The calibration presumes a sufficiently large, independent validation set, so small validation samples or dependence between training and validation can harm calibration. For Cox and other data-dependent losses, reliable nuisance estimation is required on the validation set, and the one-step debiased construction depends on the asymptotic linearity conditions. For LambdaRank, the paper advises handling ties and truncation carefully (Hines et al., 1 Jun 2026).

The recommended practice is correspondingly conservative: use an independent validation set; prefer FWD-SS or STAB-SS with $f:X\to\mathbb{R}$6–$f:X\to\mathbb{R}$7; compute nuisance estimates on validation data for data-dependent losses; monitor near-collinearity if using STAB-SS; and interpret the threshold as a scale-calibrated regularizer rather than a strict test-level $f:X\to\mathbb{R}$8. Within those constraints, the run-wise certificate supplied by ScoreStop is a principled substitute for patience counts because it is anchored to an interpretable null hypothesis and a calibrated test statistic.

6. Broader landscape and methodological contrasts

Subsequent work shows that run-wise early-stopping certificates are now used across generation, optimization, certification, routing, and quantum learning, but the stopping evidence and guarantee class vary (Akter et al., 7 Oct 2025, Xie et al., 15 Feb 2026, Li et al., 28 May 2026, Cullen et al., 26 Jun 2026, Aolaritei et al., 15 Dec 2025, Akhauri, 9 Sep 2025, Bang, 11 Feb 2026).

Setting	Certificate mechanism	Scope of guarantee
Gradient boosting	Functional score test $f:X\to\mathbb{R}$9	Per-iteration $f_t$00 calibration
LLM generation	E-process threshold $f_t$01	Anytime-valid $f_t$02-level control
Reasoning models	Maxwise conformal or renewal thresholds	False-positive halt control on well-posed queries
Projected SGD	Confidence sequence $f_t$03	$f_t$04-optimality with probability at least $f_t$05
Agentic routing	Frontier key bounded by incumbent	Deterministic replayable halting soundness

Sequential-EDFL is the clearest anytime-valid analogue. It constructs a self-normalized empirical-Bernstein e-process over clipped token-wise information lift relative to a fixed skeleton baseline, uses mixtures over $f_t$06, supports adaptive resets, and stops at the first time the e-process reaches $f_t$07. On six benchmarks it reduces generation by $f_t$08–$f_t$09 versus sequential baselines with $f_t$10 computational overhead, but the paper emphasizes that the certificate controls information sufficiency, not factual correctness; even with a correctness gate, $f_t$11 of stopped sequences remain incorrect (Akter et al., 7 Oct 2025). This is a markedly different semantic object from ScoreStop’s consistency-with-$f_t$12 certificate.

For reasoning models, uncertainty-aware stopping monitors arrivals of uncertainty keywords. The nonparametric maxwise conformal rule gives the finite-sample guarantee

$f_t$13

which the paper interprets as a run-wise early-stopping certificate with false-positive risk at most $f_t$14 on well-posed queries under exchangeability (Xie et al., 15 Feb 2026). Empirically, the keyword-based rules achieve up to $f_t$15 token savings on ill-posed math tasks while maintaining low false-positive rates.

In reinforcement learning, ESPO derives a trajectory-wise certificate from logits already computed during sampling. It forms a normalized and smoothed surrogate regret $f_t$16 and halts when

$f_t$17

Truncated trajectories are mapped to absorbing failure states with a terminal penalty, concentrating negative temporal-difference errors near the detected failure step. On DeepSeek-R1-Distill-Qwen-7B for mathematical reasoning, ESPO exceeds PPO on AIME 2024, AMC 2023, and MATH-500 while saving more than $f_t$18 rollout tokens cumulatively (Li et al., 28 May 2026).

Run-wise certificates also appear in fixed-input certification. In randomized smoothing, anytime-valid per-input stopping is obtained by inverting mixture e-processes for Bernoulli success probabilities and mapping the resulting lower confidence sequence to a certified radius $f_t$19. With meta-learned priors for the mixture, the method reports a $f_t$20-fold reduction in sample complexity relative to traditional fixed-sample randomized smoothing while maintaining rigorous statistical guarantees (Cullen et al., 26 Jun 2026). For projected SGD with general convex objectives, anytime-valid confidence sequences yield an online stopping rule $f_t$21 such that the returned weighted-average iterate is $f_t$22-optimal with probability at least $f_t$23, and the stopping time is almost surely finite under standard stochastic approximation stepsizes (Aolaritei et al., 15 Dec 2025).

Other variants are non-statistical but still run-wise. Ledger-verified routing certificates for perturb-and-MAP best-first search halt when the maximum frontier key is no larger than the incumbent realized value, with validator replay checking the frontier invariant and halting soundness under the same realized exponential race (Akhauri, 9 Sep 2025). In quantum learning under one-bit feedback, the certificate is a run-length event: halt after $f_t$24 consecutive successes under frozen control. Its usefulness depends sharply on noise, with the paper identifying the feasibility threshold $f_t$25, beyond which the expected halting time grows exponentially (Bang, 11 Feb 2026).

This broader landscape makes two distinctions especially important. First, some certificates are time-uniform and optional-stopping safe, while others are only calibrated at a fixed or per-iteration horizon. Second, the object being certified may be statistical optimality, information sufficiency, non-premature halting, frontier dominance, or robustness radius. Run-wise early-stopping certificates are therefore best treated as a methodological class defined by auditable stopping semantics, not by a single proof technique or guarantee format.