Familywise Evidence: Continuous Multiplicity Control

• Familywise evidence is a continuous generalization of classical familywise error control that quantifies the strongest evidence against true hypotheses using structured E-measures.
• The framework introduces E-measures governed by axioms like closure and implication, ensuring coherent evidence mapping even under post hoc analysis.
• On intersection-closed hypothesis classes, the least true hypothesis controls multiplicity, delivering valid evidence without additional corrections.

Familywise evidence is a continuous-evidence generalization of classical familywise error control. In this framework, the object of interest is not merely whether at least one true null hypothesis is falsely rejected, but the strongest evidence assigned to any true hypothesis in an entire hypothesis class after observing the data. The concept is introduced through the E-measure, a measure-like extension of the E-value from single hypotheses to structured classes of hypotheses, and it is designed so that post hoc inspection over a family of hypotheses remains valid in a familywise sense (Koning, 22 Apr 2026).

1. Definition and statistical target

Classical familywise inference is organized around the familywise error rate (FWER), the probability of rejecting at least one true null hypothesis. In a general multiple-testing setup with hypothesis family $\mathcal H$, true subfamily $\mathcal T(M)$, false subfamily $\mathcal F(M)$, and final rejection set $\mathcal R_\infty$, strong FWER control is written as

$$\mathrm P_M\bigl(\mathcal R_\infty\subseteq \mathcal F(M)\bigr)\ge 1-\alpha$$

for every $M\in\mathbb M$ (Goeman et al., 2012). This criterion protects the integrity of simultaneous claims by requiring that, with probability at least $1-\alpha$, all reported rejections are false nulls.

Familywise evidence replaces this binary event by a continuous statistic. For a single hypothesis $H$, an E-variable is a nonnegative measurable statistic $x\mapsto \e(H\mid x)$, valid when

$\Ex^P[\e(H \mid X)] \le 1, \quad \textnormal{for every } P \in H.$

Larger values indicate stronger evidence against $\mathcal T(M)$0. For an E-function kernel $\mathcal T(M)$1, the familywise evidence at truth $\mathcal T(M)$2 is

$\mathcal T(M)$3

and familywise evidence is controlled if

$\mathcal T(M)$4

This is the exact continuous-evidence analogue of FWER (Koning, 22 Apr 2026).

The reduction to the binary case is explicit. If $\mathcal T(M)$5 is $\mathcal T(M)$6-valued, then thresholding at $\mathcal T(M)$7 produces a rejection rule, and the familywise evidence criterion becomes classical FWER control at level $\mathcal T(M)$8 (Koning, 22 Apr 2026). Familywise evidence is therefore not merely analogous to FWER; it contains FWER as a special case.

2. E-measures and the logical structure of evidence

The formal apparatus begins with an E-function, a map

$\mathcal T(M)$9

satisfying $\mathcal F(M)$0. This encodes the Impossibility axiom: logically impossible hypotheses receive maximal evidence. The Implication axiom requires that if $\mathcal F(M)$1, then the more specific hypothesis $\mathcal F(M)$2 cannot receive less evidence against it than $\mathcal F(M)$3. This yields an E-capacity: $\mathcal F(M)$4

An E-measure strengthens this by the Closure axiom: $\mathcal F(M)$5 for every $\mathcal F(M)$6. The evidence against a union is therefore the weakest evidence against any of its constituent parts. On the reciprocal scale $\mathcal F(M)$7, this becomes a maxitive set function rather than an additive one. The paper’s characterization is that E-measures are “measure-like,” but closed under infimums instead of addition (Koning, 22 Apr 2026).

This structure depends on the hypothesis class $\mathcal F(M)$8. The class is assumed union-closed, and the strongest multiplicity results require intersection-closure. Under intersection-closure, every $\mathcal F(M)$9 has a least hypothesis

$\mathcal R_\infty$0

and every $\mathcal R_\infty$1 has the canonical representation

$\mathcal R_\infty$2

In that case the closure of an E-capacity is

$\mathcal R_\infty$3

The entire E-measure is thus determined by its values on least hypotheses (Koning, 22 Apr 2026).

The paper further shows that E-measures are the only non-dominated such objects if the hypothesis class is closed under intersections (Koning, 22 Apr 2026). This makes the least-hypothesis representation the central structural device behind familywise evidence.

3. Least true hypotheses, closure, and multiplicity

The pivotal theorem states that, on an intersection-closed $\mathcal R_\infty$4, validity of an E-capacity kernel is equivalent to familywise evidence control (Koning, 22 Apr 2026). The decisive identity is

$\mathcal R_\infty$5

Because every true hypothesis containing $\mathcal R_\infty$6 must also contain the least true hypothesis $\mathcal R_\infty$7, antitonicity implies that the maximum evidence against any true hypothesis is attained at $\mathcal R_\infty$8.

This has a distinctive multiplicity consequence. Under intersection-closure, plain hypothesis-wise validity already implies post hoc control of the strongest evidence against any true hypothesis in the family. The paper states this sharply: E-measures control familywise evidence and false evidence rate without multiplicity correction if the hypothesis class is intersection-closed (Koning, 22 Apr 2026). The reason is structural rather than Bonferroni-type: the search over all true hypotheses collapses to a single least true hypothesis.

Thresholding an E-measure at level $\mathcal R_\infty$9 yields a closed testing procedure (Koning, 22 Apr 2026). This places familywise evidence in direct relation to classical closure-based FWER logic. In the fixed-sample binary setting, closed testing itself can be viewed as a sequentially rejective procedure on the closure of $$\mathrm P_M\bigl(\mathcal R_\infty\subseteq \mathcal F(M)\bigr)\ge 1-\alpha$$0, where a hypothesis becomes testable only after all stricter implying hypotheses have been rejected (Goeman et al., 2012). The continuous E-measure formulation retains that logical organization, but it reports a full evidence map $$\mathrm P_M\bigl(\mathcal R_\infty\subseteq \mathcal F(M)\bigr)\ge 1-\alpha$$1 rather than only reject/non-reject outcomes.

This suggests a reinterpretation of multiplicity: in logically structured classes, multiplicity is not handled by shrinking each individual claim through a universal penalty, but by exploiting the order induced by implication and intersection. In the E-measure framework, that order is encoded directly in the object that represents evidence.

4. False evidence rate and general false-evidence functionals

Familywise evidence is the max-type criterion. The corresponding average-type criterion is the false evidence rate (FER), introduced as a continuous-evidence analogue of false discovery rate (Koning, 22 Apr 2026).

Given a finite selection rule $$\mathrm P_M\bigl(\mathcal R_\infty\subseteq \mathcal F(M)\bigr)\ge 1-\alpha$$2, the false evidence proportion at truth $$\mathrm P_M\bigl(\mathcal R_\infty\subseteq \mathcal F(M)\bigr)\ge 1-\alpha$$3 is

$$\mathrm P_M\bigl(\mathcal R_\infty\subseteq \mathcal F(M)\bigr)\ge 1-\alpha$$4

and the false evidence rate is

$$\mathrm P_M\bigl(\mathcal R_\infty\subseteq \mathcal F(M)\bigr)\ge 1-\alpha$$5

The corresponding false selection proportion is

$$\mathrm P_M\bigl(\mathcal R_\infty\subseteq \mathcal F(M)\bigr)\ge 1-\alpha$$6

The key bound is

$$\mathrm P_M\bigl(\mathcal R_\infty\subseteq \mathcal F(M)\bigr)\ge 1-\alpha$$7

again showing that the least true hypothesis controls the full familywise behavior (Koning, 22 Apr 2026). Two corollaries follow. First, for a fixed selection rule, controlling $$\mathrm P_M\bigl(\mathcal R_\infty\subseteq \mathcal F(M)\bigr)\ge 1-\alpha$$8 suffices for FER control. Second, on intersection-closed classes, validity is equivalent to uniform FER control across all selection rules (Koning, 22 Apr 2026).

The paper also embeds familywise evidence and FER in a broader framework based on functionals $$\mathrm P_M\bigl(\mathcal R_\infty\subseteq \mathcal F(M)\bigr)\ge 1-\alpha$$9. Validity under $M\in\mathbb M$0 means

$M\in\mathbb M$1

When $M\in\mathbb M$2 is local, positively homogeneous, and monotonic, one has

$M\in\mathbb M$3

Taking $M\in\mathbb M$4 recovers familywise evidence; taking

$M\in\mathbb M$5

recovers FER (Koning, 22 Apr 2026). The same least-hypothesis mechanism therefore governs a broad class of false-evidence criteria.

5. Sequential, adaptive, and online relatives of familywise evidence

Familywise evidence is formulated as a continuous generalization of familywise error control, and the surrounding literature clarifies how family-level validity behaves in binary sequential and adaptive settings. In fixed-sample multiple testing, the sequential rejection principle states that strong FWER control follows from a monotonicity condition on the rejection rule together with a single-step safety condition in the configuration where all already rejected hypotheses are false (Goeman et al., 2012). For streaming data, the rejection principle is extended by allowing the rejection function to depend on previously rejected hypotheses, previously accepted hypotheses, and current sampling state, thereby handling the specifically sequential distinction between “accepted,” “rejected,” and “still active” (Bartroff et al., 2013).

Adaptive data analysis introduces a different difficulty: the analyst may change the testing strategy after inspecting the data. The interactive FWER test addresses this by masking each $M\in\mathbb M$6-value into a revealed part $M\in\mathbb M$7 and a hidden bit $M\in\mathbb M$8, then stopping when

$M\in\mathbb M$9

falls below $1-\alpha$0 (Duan et al., 2020). The formal guarantee is FWER control under independence of null $1-\alpha$1-values and independence between nulls and non-nulls. The procedure is explicitly designed so that a human or computer program may adaptively guide the testing strategy while retaining familywise validity (Duan et al., 2020).

Online testing introduces an a priori unbounded sequence of hypotheses. Online Sidak,

$1-\alpha$2

online fallback,

$1-\alpha$3

and ADDIS-Spending,

$1-\alpha$4

all provide strong familywise guarantees under their respective independence or local-dependence assumptions (Tian et al., 2019). The paper proves these procedures via PFER control, using the implication $1-\alpha$5 (Tian et al., 2019).

These developments remain binary rather than continuous, but they exhibit the same underlying concern: family-level validity must survive post hoc choice, sequential updating, and structural dependence. A plausible implication is that familywise evidence extends this logic from rejection events to full evidence surfaces, while preserving the family-level guarantee through the least true hypothesis.

6. Scope, extensions, and limitations

The strongest familywise-evidence results require that $1-\alpha$6 be intersection-closed, that $1-\alpha$7 be an E-capacity kernel or its closure to an E-measure, and that the relevant suprema and infima be measurable (Koning, 22 Apr 2026). Intersection-closure is not a cosmetic assumption: it is what guarantees existence of $1-\alpha$8 and makes the identity

$1-\alpha$9

available. Without least hypotheses, the argument that ordinary validity already controls post hoc familywise evidence does not go through in the same way.

The framework also extends beyond standard hypothesis testing. In decision-theoretic form, the paper proves a uniform E-consequence bound

$H$0

which is structurally analogous to familywise evidence but indexed by decisions rather than hypotheses (Koning, 22 Apr 2026). The same paper also advocates updating from E-prior to E-posterior and extending the framework to predictive E-measures (Koning, 22 Apr 2026).

By contrast, related binary-control frameworks often require stronger probabilistic assumptions. Step-up procedures based on Simes’ inequality need independence or certain positive dependence structures (Goeman et al., 2012). Interactive masking procedures assume null $H$1-values are mutually independent and independent of the non-nulls (Duan et al., 2020). Online adaptive-discarding procedures require independence or local dependence and, for the strongest guarantees, uniformly conservative null $H$2-values (Tian et al., 2019). These comparisons do not weaken the familywise-evidence construction; rather, they clarify that its “no multiplicity correction” phenomenon is specifically structural and tied to intersection-closed hypothesis classes.

In summary, familywise evidence reframes multiplicity from a problem of adjusting isolated $H$3-values to a problem of representing evidence coherently across a logically structured class of hypotheses. Its central object is the E-measure, its central mechanism is the least true hypothesis $H$4, and its central claim is that, on intersection-closed classes, ordinary validity already yields post hoc familywise protection in a continuous-evidence sense (Koning, 22 Apr 2026).