---
title: Sequential Hypothesis Testing E-valuators
url: https://www.emergentmind.com/topics/sequential-hypothesis-testing-based-e-valuators
type: topic
---

# Sequential Hypothesis Testing E-valuators

Sequential hypothesis testing-based e-valuators are anytime-valid statistical procedures that evaluate hypotheses using streams of data by dynamically updating a process—usually a nonnegative supermartingale—constructed from so-called e-values or e-variables. These methods allow valid inference under optional stopping, adversarial data collection, and adaptivity, and are particularly powerful for nonparametric, composite, and online testing tasks. The e-value paradigm unifies testing, confidence sequences, and log-optimal betting strategies, with foundational results ensuring rigorous Type I error control, compatibility with online convex optimization, and systematic construction schemas for a wide variety of testing problems.

## 1. Mathematical Foundations of Sequential E-valuators

Let $\mathcal{P}$ denote a family of distributions over a sample space $\mathcal{X}$ and let $T:\mathcal{P}\rightarrow\Theta\subset\mathbb{R}^d$ be an elicitable and identifiable functional of interest (e.g., mean, quantile, expectile), specified via a strictly consistent scoring rule $S:\Theta\times\mathcal{X}\rightarrow\mathbb{R}$ such that, for all $P\in\mathcal{P}$, the population risk 
$$\theta\mapsto \mathbb{E}_{X\sim P}S(\theta,X)$$
is uniquely minimized at $\theta=T(P)$ [2204.05680]. The identification function $V(\theta,x)=\nabla_\theta S(\theta,x)$ links the scoring rule to a vector-valued root characterization: $T(P)$ satisfies $\mathbb{E}_{X\sim P}[V(\theta,X)]=0$.

A sequential e-value process (or e-process) is a nonnegative adapted process $(M_n)_{n\ge 0}$ in a filtration $(\mathcal{F}_n)_{n\ge 0}$ such that, under the null hypothesis $H_0:\ T(P) = \theta_0$,
$$\mathbb{E}[M_n\mid\mathcal{F}_{n-1}]\le M_{n-1},\qquad M_0=1.$$
The one-step "betting" increments are constructed as
$$m_n = 1 + \eta_n V(\theta_0, X_n),$$
with $\eta_n$ a predictable sequence chosen to ensure $m_n\ge 0$ ($\eta_n V(\theta_0,X_n) > -1$). The process $M_n = \prod_{i=1}^n m_i$ is a nonnegative supermartingale and thus a valid e-process [2204.05680, 2410.23614].

More generally, for any sequential e-variables $E_n$ adapted to $\mathcal{F}_{n}$ with $\mathbb{E}[E_n|\mathcal{F}_{n-1}]\le 1$, the product $M_n = \prod_{i=1}^n E_i$ forms an e-process [2109.11761]. This construction is central to the sequential hypothesis testing-based e-valuator paradigm.

## 2. Anytime-Validity and Type I Error Control

Sequential e-valuators provide rigorous Type I error control at any stopping time via Ville's inequality:
$$
\mathbb{P}_{H_0}\left(\sup_{n\ge 0}M_n\ge 1/\alpha\right) \le \alpha
$$
for any $\alpha\in(0,1)$ [2204.05680, 2410.23614, 2109.11761]. Consequently, the stopping rule
$$
\tau = \inf\{n\ge 1 : M_n \ge 1/\alpha\}
$$
defines an $\alpha$-level anytime-valid sequential test. This property is robust to optional stopping, stopping for a reason, data-adaptive stopping, and even certain forms of adversarial data collection.

In composite or nonparametric settings, predictable mixture strategies over a family $\{L^\theta\}_{\theta\in\Theta}$ of base supermartingales or e-values are often employed. Here, the e-process is
$$M_n = \prod_{i=1}^n \int_\Theta \frac{L_i^\theta}{L_{i-1}^\theta}\,\pi_i(d\theta),$$
with $\pi_i$ a $\mathcal{F}_{i-1}$-measurable probability measure over $\Theta$ [2204.05680]. Mixture-of-experts or online-convex-optimization (OCO) strategies select the mixing measure to adaptively maximize expected log-growth or minimize regret.

## 3. Design and Algorithmic Structure

The sequential e-valuator framework is algorithmically instantiated as follows [2204.05680, 2410.23614]:
1. **Initialization:** Set $M_0=1$ and choose target significance $\alpha$.
2. **Observation:** For $n=1,2,\ldots$, observe new data $X_n$.
3. **Adaptation:** Compute $\eta_n$ (or the parameter $\pi_n$ for mixture strategies) as a function of $X_1,\ldots,X_{n-1}$.
4. **Increment Computation:**
   $$
   m_n = 1 + \eta_n V(\theta_0, X_n) \quad \text{or} \quad m_n = \int_\Theta \frac{L_n^\theta}{L_{n-1}^\theta}\,\pi_n(d\theta).
   $$
5. **Update:** $M_n = M_{n-1}m_n$.
6. **Stopping Rule:** Halt and reject $H_0$ if $M_n \ge 1/\alpha$.

In higher-dimensional or non-bounded data settings, sub-$\psi$ cumulant generating function bounds allow construction of exponential supermartingales:
$$
L_n(u) = \exp\left(u Y_n - \sum_{i=1}^n \psi(u)\right), \qquad Y_n = \sum_{i=1}^n \Delta Y_i, \ \Delta Y_i = V(\theta_0, X_i),
$$
and predictable mixtures over $u$ and $\theta$ yield e-processes robust to sub-exponential or heavy-tailed data [2204.05680].

For broad composite nulls, the family of e-processes $M_n^{(\theta)}$ is often mixed with respect to a prior or adapted with online learning [2504.02818, 2410.23614]. Portfolios of e-processes with sublinear "regret" versus the oracle best in hindsight yield universal log-optimality [2504.02818].

## 4. Power Analysis and Regret Bounds

Under composite alternatives, sequential e-valuators achieve exponential power growth if there exists $\theta^*$ such that
$$
\liminf_{n\to\infty}\tfrac1n \log L_n^{\theta^*} > 0\quad \text{a.s.}
$$
and if the regret,
$$
\mathrm{Regret}_n = \max_{\theta\in\Theta}\left(\log L_n^\theta - \log M_n\right), \qquad \text{(for mixture $M_n$)}
$$
is $o(n)$ almost surely. Results from OCO (e.g., Follow-the-Leader, FTRL, OGD) guarantee sublinear regret and thus asymptotic power one [2204.05680, 2504.02818]:

- **Asymptotic Growth:** For log-optimal betting strategies,
  $$
  \lim_{n\to\infty} \frac{1}{n} \log M_n = \ell^*_Q > 0
  $$
  almost surely under the alternative $Q$, with $\ell^*_Q$ the maximal expected log-increment [2504.02818].

- **Stopping-Time Efficiency:** The expected stopping time under any $Q$ satisfies
  $$
  \lim_{\alpha\to 0} \frac{\mathbb{E}_Q[\tau_\alpha]}{\log(1/\alpha)} = 1/\ell^*_Q,
  $$
  i.e., the test achieves optimal expected sample complexity, matching the pointwise Kelly rate [2504.02818].

## 5. Applicability and Scope

Sequential hypothesis testing-based e-valuators, as developed in [2204.05680], cover
- **Means and mean ratios** under boundedness or sub-Gaussian/sub-exponential conditions,
- **Quantiles and expectiles** including VaR, multi-quantile vectors, and regression functionals,
- **Goodness-of-fit, forecast calibration**, and **probabilistic model assessment** in nonparametric setups [2109.11761, 2103.08402],
- **High-dimensional and ML-driven tests** (e.g., deep classification two-sample, online independence and conditional independence) with nonparametric power via learned betting strategies [2310.19384, 2210.13027].

The framework is also extensible to online and multiple testing settings—where e-values per hypothesis are multiplied or combined via closed testing or "discovery matrix" procedures for strong FWER/FDP control [2407.15733, 2003.00593, 2501.09015].

## 6. Robustness, Extensions, and Practical Considerations

Robustness to unbounded or heavy-tailed data is handled via sub-$\psi$ bounds; e-processes can be constructed in exponential form and mixtures over parameter or tuning spaces can be performed using online convex optimization [2204.05680]. The same formalism admits sequential testing with estimation, confidence sequences, and stopping-time statements.

Advantages include:
- **Anytime-valid error control:** Type I error control is uniform over all (possibly data-adaptive) stopping rules;
- **Direct interpretability of evidence:** The process $M_n$ can be regarded as accumulated "betting capital" against $H_0$;
- **Flexible, adaptive power optimization:** OCO or hedged mixtures allow minimax or log-optimal adaptivity over alternatives;
- **Compatibility with decision-theoretic and modern ML protocols,** supporting test design via virtually any scoring rule or identification functional.

Prominent limitations and considerations are
- The requirement that the null functional be explicitly elicitable and have a strict identification function.
- Power may be below fixed-sample tests in specific dependence structures or high-lag blocking [2204.05680, 2103.08402].
- Tuning of betting fractions or mixture priors is critical for power; algorithmic learnability conditions are needed for some log-optimality results.

For particular tasks such as sequential forecast calibration or nonparametric two-sample testing, specialized construction and batching strategies provide empirical evidence of strong power and low Type I error in both simulated and real-data settings [2109.11761, 2210.13027].

## 7. Connections and Impact

This methodology unifies frequentist and Bayesian perspectives: e-values generalize p-values (via $1/E$ calibration) but permit optional continuation and combination; in the simple hypothesis case, e-values are likelihood ratios and coincide with Bayes factors under suitable priors [2410.23614, 1906.07801]. The interpretation via test supermartingales ties e-valuator procedures to game-theoretic probability and log-optimal portfolio constructions.

Sequential e-valuators underpin modern approaches to universal inference, multistage experimental design, adaptive clinical trials, online learning, and real-time decision systems. Their generality, statistical safety, and universality under optional stopping make them central tools for rigorous sequential inference in contemporary statistics and machine learning [2204.05680, 2410.23614, 2504.02818, 2310.19384].

Source: https://www.emergentmind.com/topics/sequential-hypothesis-testing-based-e-valuators