Valid in Every Hypothesis (VEH)

• VEH is a uniform validity criterion that guarantees statistical tests and robot trajectories remain valid across every hypothesis in a given null set.
• It underpins anytime-valid inference by enforcing e-value and e-process conditions, allowing sequential monitoring with robust type-I error control.
• In kinodynamic motion planning, VEH demands conservative feasibility by requiring trajectory paths to be valid in every hypothesized world model.

Valid in Every Hypothesis (VEH) is a uniform validity requirement that appears most prominently in modern hypothesis testing with e-values and e-processes. In that setting, VEH means that the defining validity inequality holds for every distribution in a null set, not merely on average over the null or only at a pre-fixed sample size. This makes VEH central to composite-null testing, optional stopping, and anytime-valid inference, including testing-by-betting constructions based on non-negative martingales or supermartingales (Ramdas et al., 2024). The same acronym is also used in a distinct sense in kinodynamic motion planning, where a trajectory is required to be feasible in every hypothesis of a robot’s recent world-model history (Damm et al., 30 Sep 2025).

1. Definition and formal statistical meaning

For a composite null hypothesis represented by a set of distributions $\mathcal{P}$, an e-variable $E$ is VEH if

$$\mathbb{E}^{P}[E] \leq 1 \quad \text{for all } P \in \mathcal{P}.$$

In the formulation used for e-values, this is the basic definition of uniform validity under composite nulls (Ramdas et al., 2024). For simple nulls, where $\mathcal{P}=\{P\}$, the definition reduces to the classical one-distribution case.

Parallel definitions extend the same principle to tests and p-variables. A test $\phi$ is VEH if, for all $P \in \mathcal{P}$,

$\mathbb{E}^{P}[\phi] \leq \alpha,$

and a p-variable is VEH if, for all $P \in \mathcal{P}$,

$\mathbb{P}^{P}(p \leq \alpha) \leq \alpha.$

For e-variables, Markov’s inequality yields the basic rejection guarantee

$\mathbb{P}^{P}(E \geq 1/\alpha) \leq \alpha,$

again for every $E$0 (Ramdas et al., 2024).

In sequential settings, the same idea becomes a stopping-time statement: an e-process $E$1 is VEH when

$E$2

for every $E$3 and any stopping time $E$4 (Ramdas et al., 2024). In supervised prediction, an equivalent operational form is called uniform validity; it permits simultaneous control even if the asserted event is data-dependent or adversarially chosen (Cella et al., 2021).

VEH is therefore stronger than validity statements tied to a single fixed analysis plan. One formulation states explicitly that this is stronger than typical p-value-based validity, which is only for the whole null hypothesis as a set, not at the level of individual distributions (Arnold et al., 2021).

2. E-values, e-processes, and optional stopping

An e-value is a non-negative random variable $E$5 such that, under a null hypothesis $E$6,

$E$7

In sequential settings with filtration $E$8 and lag $E$9, the adapted version is

$$\mathbb{E}^{P}[E] \leq 1 \quad \text{for all } P \in \mathcal{P}.$$0

which is the basic e-value condition used in sequential forecast calibration (Arnold et al., 2021).

For $$\mathbb{E}^{P}[E] \leq 1 \quad \text{for all } P \in \mathcal{P}.$$1, e-values combine multiplicatively: $$\mathbb{E}^{P}[E] \leq 1 \quad \text{for all } P \in \mathcal{P}.$$2 For $$\mathbb{E}^{P}[E] \leq 1 \quad \text{for all } P \in \mathcal{P}.$$3, sequential validity is retained through a U-statistic average,

$$\mathbb{E}^{P}[E] \leq 1 \quad \text{for all } P \in \mathcal{P}.$$4

where

$$\mathbb{E}^{P}[E] \leq 1 \quad \text{for all } P \in \mathcal{P}.$$5

Large e-values provide evidence against the null, and the multiplicative inverse of an e-value is a conservative p-value (Arnold et al., 2021).

The testing-by-betting formulation gives a particularly explicit VEH mechanism. In deep anytime-valid testing, for each batch $$\mathbb{E}^{P}[E] \leq 1 \quad \text{for all } P \in \mathcal{P}.$$6 and network $$\mathbb{E}^{P}[E] \leq 1 \quad \text{for all } P \in \mathcal{P}.$$7, the betting score is

$$\mathbb{E}^{P}[E] \leq 1 \quad \text{for all } P \in \mathcal{P}.$$8

the wealth update is

$$\mathbb{E}^{P}[E] \leq 1 \quad \text{for all } P \in \mathcal{P}.$$9

and the procedure rejects $\mathcal{P}=\{P\}$0 at the first $\mathcal{P}=\{P\}$1 such that

$\mathcal{P}=\{P\}$2

Under $\mathcal{P}=\{P\}$3, $\mathcal{P}=\{P\}$4 is a non-negative martingale, so Ville’s inequality gives

$\mathcal{P}=\{P\}$5

This yields finite-sample type I error control under continuous monitoring and optional stopping; the paper identifies this strong form of validity as the VEH principle (Pandeva et al., 2023).

The core consequence is that a practitioner does not need to pre-specify a sample size. Evidence can be monitored continuously, and rejection remains valid at the first threshold crossing.

3. VEH in deep anytime-valid nonparametric testing

A general nonparametric framework expresses the null through two operators $\mathcal{P}=\{P\}$6 and $\mathcal{P}=\{P\}$7 acting on the data distribution $\mathcal{P}=\{P\}$8: $\mathcal{P}=\{P\}$9 This operator-action formulation is used to unify two-sample testing, independence testing, conditional independence, group invariance, and adversarial robustness in machine learning (Pandeva et al., 2023).

A distinctive feature of the framework is that the payoff function and the betting strategy are not decoupled. Instead, both are learned jointly with a deep neural network by maximizing log-wealth: $\phi$0 The stopping time

$\phi$1

is a level-$\phi$2 test of power one, with

$\phi$3

provided the learning procedure is sufficiently consistent (Pandeva et al., 2023).

The framework is presented as a unified machinery for designing and implementing anytime-valid tests across classical and modern tasks. Reported experiments include synthetic and real data such as adversarial CIFAR-10, MNIST rotation, and the Blob dataset, with comparisons to E-C2ST, Seq-IT, MMD, and S-C2ST. The reported findings are tighter or faster type I error control under continuous monitoring, better sample efficiency through early stopping, and competitive or superior power when equipped with powerful DNNs (Pandeva et al., 2023).

A common misunderstanding is to equate these guarantees with ordinary fixed-sample validity. The stated guarantee is stronger: the probability of a false rejection never exceeds $\phi$4, no matter how the data are monitored or when the test is stopped (Pandeva et al., 2023).

4. Post-hoc significance levels, admissibility, and the necessity of e-values

VEH has also become central in post-hoc hypothesis testing, where the significance level may be chosen after seeing and analyzing the data. In this literature, the analyst or adversary selects a loss function, or equivalently a significance level, through a measurable mapping $\phi$5. The adversary set $\phi$6 specifies which such mappings are allowed. Two extreme cases are $\phi$7, all possible measurable mappings, and $\phi$8, only constant mappings (Chugg et al., 1 Aug 2025).

Within this framework, $\phi$9-admissibility corresponds to VEH: the procedure remains type-I risk safe for arbitrary post-hoc, data-dependent choices of the loss or significance level (Chugg et al., 1 Aug 2025). The central characterization is that admissible VEH procedures must be based on e-values. For a test family $P \in \mathcal{P}$0, the associated e-variable is

$P \in \mathcal{P}$1

Canonical representations are

$P \in \mathcal{P}$2

for randomized tests and

$P \in \mathcal{P}$3

for binary tests. Theorems summarized in the paper state that, under the stated conditions, $P \in \mathcal{P}$4-admissible tests are exactly those defined by sharp e-values (Chugg et al., 1 Aug 2025).

A related 2026 treatment asks how classical type-I error should be extended to data-dependent levels and concludes that the correct extension is control of the expected distortion ratio,

$P \in \mathcal{P}$5

This criterion is characterized by three axioms (Koning, 27 May 2026):

• Nesting classical validity: if $P \in \mathcal{P}$6 is constant, the extension coincides with classical validity.
• Preserving classical validity: the data-dependent rule must not violate fixed-threshold error control.
• Monotonicity: making the rejection claim weaker cannot invalidate the procedure.

The same work connects this criterion directly to VEH by showing that the numerical representation of the generalized test is an e-value satisfying

$P \in \mathcal{P}$7

for every $P \in \mathcal{P}$8 in the null (Koning, 27 May 2026).

Taken together, these results establish a precise boundary: for the most flexible post-hoc guarantee, e-value-based procedures are not merely convenient but necessary.

5. Extensions across forecasting, prediction, and watermarking

VEH has been instantiated in several domains beyond abstract hypothesis testing. The common structure is a validity inequality that holds for every null distribution or every member of a null family, often together with optional stopping.

Setting	VEH condition	Consequence
Forecast calibration (Arnold et al., 2021)	$P \in \mathcal{P}$9 for all $\mathbb{E}^{P}[\phi] \leq \alpha,$0	Sequential goodness-of-fit testing with optional stopping
Probability forecast performance (Henzi et al., 2021)	$\mathbb{E}^{P}[\phi] \leq \alpha,$1 for all $\mathbb{E}^{P}[\phi] \leq \alpha,$2	Finite-sample validity without assumptions on the data generating processes
Supervised prediction (Cella et al., 2021)	$\mathbb{E}^{P}[\phi] \leq \alpha,$3	Uniform validity for probabilistic predictors; valid predictors must be imprecise
Statistical watermarking (Huang et al., 19 Feb 2026)	$\mathbb{E}^{P}[\phi] \leq \alpha,$4	Anytime-valid watermark detection via a test supermartingale

In sequential forecast calibration, e-values are constructed from PITs or ranks, and rejection occurs when the cumulative e-process crosses $\mathbb{E}^{P}[\phi] \leq \alpha,$5; the same framework is also described as more generally applicable for sequential goodness-of-fit testing (Arnold et al., 2021). For forecast comparison using proper scoring rules, the resulting e-values are valid in finite samples without any assumptions on the data generating processes, and optional stopping is explicitly allowed (Henzi et al., 2021).

In supervised learning, the corresponding notion is a valid probabilistic predictor. The papers state that valid probabilistic predictors must be imprecise and that uniform validity yields frequentist error-rate control for derived tests and prediction sets (Cella et al., 2021). In this setting, VEH is formulated through plausibility contours and the probability of under-rating the true future observation.

In statistical watermarking for LLMs, Anchored E-Watermarking constructs a test supermartingale for detection. The detector uses an anchor distribution $\mathbb{E}^{P}[\phi] \leq \alpha,$6, requires uniform e-value validity over $\mathbb{E}^{P}[\phi] \leq \alpha,$7, and monitors

$\mathbb{E}^{P}[\phi] \leq \alpha,$8

with rejection at the first $\mathbb{E}^{P}[\phi] \leq \alpha,$9 such that $P \in \mathcal{P}$0. Reported evaluations state that the framework reduces the average token budget required for detection by 13–15% relative to state-of-the-art baselines (Huang et al., 19 Feb 2026).

These examples show that VEH is not tied to a single test construction. It is a criterion for validity under composite nulls, sequential monitoring, or model uncertainty, and different domains instantiate that criterion with different e-values and stopping rules.

6. Distinct usage in kinodynamic motion planning

Outside statistical inference, VEH is also the name of a conservative planning paradigm for mobile robots navigating across inconsistent world models. Here, each “hypothesis” is one temporally sampled instance of the environment representation, and a path is considered feasible only if it is valid in every hypothesis (Damm et al., 30 Sep 2025).

The baseline formulation extends A*/ARA* over the KEASL lattice so that an edge is expanded only if it is collision-free in every stored world hypothesis. The key validity check is

$P \in \mathcal{P}$1

This makes VEH maximally conservative: it effectively treats the union of obstacles across the recent history as obstacles for planning (Damm et al., 30 Sep 2025).

The same paper contrasts VEH with PEH, GEH, and GEGRH. PEH invokes a sub-search for every node expansion that crosses a divergence point in the world models. GEH and GEGRH defer the sub-search until after an edge expands into the goal region, and GEGRH additionally revises the graph based on divergent nodes in each world (Damm et al., 30 Sep 2025).

Field experiments on 221 planning problems in an off-road environment found that PEH and GEH were impractical for deployment, while GEGRH achieved lower-cost trajectories and faster average planning times than VEH. The reported average planning times were $P \in \mathcal{P}$2 seconds for VEH versus $P \in \mathcal{P}$3 for GEGRH with two hypotheses, and $P \in \mathcal{P}$4 versus $P \in \mathcal{P}$5 with three hypotheses. The corresponding path durations were $P \in \mathcal{P}$6 versus $P \in \mathcal{P}$7 for two hypotheses and $P \in \mathcal{P}$8 versus $P \in \mathcal{P}$9 for three hypotheses (Damm et al., 30 Sep 2025).

This robotic meaning differs from the statistical one in object and mechanism, but not in its basic intuition. In both cases, VEH is a worst-case validity requirement across a family of hypotheses: null distributions in statistics, world models in navigation.