Effective Historical Sample Size

• Effective Historical Sample Size is the conversion of historical, prior, or correlated information into the number of independent observations needed to achieve comparable precision.
• Various methodologies such as variance matching, predictive consistency, and hypothesis testing are used to quantify EHSS across Bayesian borrowing, adaptive trials, and privacy-constrained settings.
• EHSS plays a crucial role in calibrating historical information, ensuring that dependence, prediction uncertainty, and design adaptations are accurately reflected in effective data contributions.

Searching arXiv for the cited papers and related EHSS work to ground the article. Effective historical sample size (EHSS) is the representation of historical, prior, weighted, or correlated information in units of hypothetical independent observations. Across Bayesian borrowing, adaptive hybrid trials, dependence-adjusted inference, privacy-constrained learning, and prediction uncertainty, the common objective is to replace a complex information source by an observation-equivalent quantity. The resulting number is not model-free: it depends on the estimand, the likelihood, the dependence structure, the borrowing rule, and, in some settings, the decision problem itself. In that sense, EHSS is less a single statistic than a family of information-equivalence constructions (Li et al., 2024, Neuenschwander et al., 2019, Mansouri, 15 May 2026).

1. Conceptual scope and unifying idea

The most stable formulation of EHSS is the “equivalent observations” view: the effective sample size is the number of hypothetical observations that would provide the same uncertainty, precision, or inferential effect as the object being summarized. Under the Gaussian approximation for EVSI, the prior for a scalar parameter is written as $\phi \sim N\!\left(\mu_0,\sigma^2/n_0\right)$, and $n_0$ is interpreted as the number of hypothetical observations yielding the same uncertainty as that distribution. In Bayesian trial design, the same idea is applied to priors derived from historical studies; in correlated-data settings, it becomes the number of independent units represented by a dependent sample; in individual prediction, it becomes the number of similar patients a prediction is effectively based on (Li et al., 2024, Thomassen et al., 2023, Bartoszek, 2015).

Several literatures make this equivalence explicit but attach it to different targets. In differentially private CVaR optimization, the informative part of a dataset of size $n$ is not all $n$ records but roughly the worst $\tau$-fraction, giving an effective tail sample size $n\tau$, and under privacy the effective private tail sample size becomes $\varepsilon n\tau$ (Mansouri, 15 May 2026). In phylogenetic comparative methods, the nominal number of species is reduced by shared ancestry, so ESS measures the number of independent species implied by the covariance structure (Bartoszek, 2015). In functional spatial statistics, ESS measures the effective number of independent curves implied by the trace-covariogram (Alegría et al., 28 Jan 2026). In importance sampling, by contrast, the widely used $\widehat{\mathrm{ESS}}=1/\sum \bar w_n^2$ is criticized precisely because it is only a weight-dispersion proxy and not a faithful approximation to the variance-based definition (Tang et al., 2018).

A plausible implication is that EHSS should always be read together with the inferential task it is meant to support. A prior may have one effective size for posterior precision, another for hypothesis testing, and a third for prediction or edge recovery. That task dependence is explicit in p-value-based ESS, predictive-consistency ESS, and graphical-model design analysis (Wang et al., 22 Jul 2025, Neuenschwander et al., 2019, Arena et al., 21 Jun 2026).

2. Formal definitions and information-equivalent constructions

A large class of EHSS definitions is built from variance or information matching. Under the Gaussian approximation to EVSI, the prior ESS $n_0$ is recovered from variance ratios such as

$$\hat n_0 = n \left( \frac{\mathrm{Var}_\phi[\phi]}{\mathrm{Var}_{\mathbf{X}_n}\!\left[\mathbb{E}_\phi(\phi\mid\mathbf{X}_n)\right]} - 1 \right),$$

or from the variance of a summary statistic. A nonparametric regression estimator replaces repeated posterior computation by regressing $n_0$0 on a low-dimensional summary $n_0$1 and estimating the variance of the posterior mean through fitted values, thereby preserving the “equivalent number of participants” interpretation while reducing computational cost (Li et al., 2024).

A second line of work emphasizes predictive consistency. For a scalar parameter with sampling Fisher information $n_0$2 and prior information $n_0$3, the expected local-information-ratio ESS is

$n_0$4

Its defining property is

$n_0$5

so that, on average under the prior predictive, a prior worth $n_0$6 observations behaves additively with $n_0$7 new observations. In the paper’s examples, competing ESS definitions can diverge substantially in non-conjugate settings, whereas ELIR satisfies the predictive consistency criterion (Neuenschwander et al., 2019).

A third construction embeds ESS in hypothesis testing. The p-value-based method compares a Bayesian $n_0$8 under an informative prior with a frequentist $n_0$9 under a hypothetical sample size $n$0, and chooses $n$1 by minimizing

$n$2

Depending on whether the truth lies under $n$3 or $n$4, ESS is defined as either $n$5 or $n$6. This construction permits negative ESS when the prior is harmful in the relevant testing context and avoids specification of a separate baseline prior (Wang et al., 22 Jul 2025).

For multivariate priors in Gaussian graphical models, ESS is defined through determinant ratios of information or variance matrices. If $n$7 and $n$8 are positive definite $n$9 matrices, with $n$0, the global ESS is

$n$1

and a parameterwise ESS is obtained through Cholesky factorization. This provides both a global observation-equivalent interpretation and an edge- or variance-specific decomposition (Arena et al., 21 Jun 2026).

The resulting formulations differ in what they preserve:

Setting	Quantity treated as effective size	Representative expression
EVSI / Gaussian approximation	Equivalent participants in prior	$n$2
Robust commensurate priors	Equivalent new-trial patients from prior precision	$n$3
Private CVaR learning	Informative tail records / private tail records	$n$4, $n$5
Functional spatial data	Effective number of independent curves	$n$6

This variety suggests that EHSS is best understood as a calibrated reduction from a complex information object to an observation-equivalent scale, rather than as a universal scalar invariant across models.

3. Historical borrowing in Bayesian design

In Bayesian clinical trial design, EHSS is most directly associated with informative priors built from historical studies. Under robust commensurate priors with multiple historical sources, each source $n$7 contributes predictive precision $n$8, and the proposed Winkler-type aggregation yields

$n$9

This makes total prior information additive and leads to

$\tau$0

The same framework derives the sample size relation

$\tau$1

so EHSS is exactly the reduction in required current-trial sample size induced by the prior. Because the mapping from discrepancy weights $\tau$2 to $\tau$3 is nonlinear, a linearization is introduced so that elicited weights correspond approximately to fractions of maximal EHSS. In the Alzheimer’s example with seven historical trials, the resulting prior variance $\tau$4 implied a total EHSS of about 160, consistent with a reduction from $\tau$5 to $\tau$6 (Whitehead et al., 2024).

A complementary Bayesian perspective asks what a prior is “worth” in a predictively consistent sense. For historical placebo information synthesized through a MAP prior in ankylosing spondylitis, ELIR ESS values depended on how accurately the MAP prior was approximated: a single Beta fit gave ESS about 26, whereas two- and three-component Beta mixtures gave ESS about 36–38. The latter values are interpretable as an effective historical control size, despite the much larger raw historical sample, because heterogeneity reduces the amount of transferable information (Neuenschwander et al., 2019).

The same logic has been extended beyond scalar or generalized-linear settings to Gaussian graphical models. There, Wishart and G-Wishart priors elicited from previous studies can be assigned prior ESS through variance-ratio, precision-ratio, MTM, PT, and ELIR constructions. The methods differ systematically: MTM, PT, and ELIR depend only on degrees of freedom, whereas VR and PR also depend on graph density and geometry. The resulting ESS values can then be used in two planning tools: the Data-to-Prior Information Ratio, which identifies sample sizes at which the data dominate the prior, and a Bayes Factor Design Analysis, which identifies sample sizes needed for conclusive edge-wise evidence (Arena et al., 21 Jun 2026).

4. Adaptive, weighted, and privacy-limited historical information

EHSS need not be fixed at design. In Bayesian hybrid trials with an interim analysis, the historical control prior is dynamically resized using a normalized Hellinger distance between the historical prior and the interim posterior for the concurrent control arm. A similarity parameter

$\tau$7

governs both stage-2 control allocation and the final prior variance. The final control prior is explicitly rescaled so that its ESS equals

$\tau$8

Thus the EHSS is literally the number of concurrent controls replaced by historical information, shrinking toward zero when historical and concurrent controls disagree and increasing when they align (Ratta et al., 22 Dec 2025).

A related but frequentist construction arises in hybrid-control designs using inverse probability weighting. Historical controls are reweighted by the probability of belonging to the current versus historical study, and the effective information contributed by those reweighted controls is governed by the standard weighted-sample formula

$\tau$9

Although not named EHSS in the original formulation, the method’s blinded sample size re-estimation rules are driven by variance inflation factors proportional to $n\tau$0. When baseline covariates differ substantially between historical and current controls, the weights become variable, $n\tau$1 drops, and the current-trial sample size is increased to compensate for the loss of effective borrowing (Kojima et al., 18 Jun 2025).

The notion also extends to privacy-constrained use of historical records. In differentially private CVaR optimization, only the worst $n\tau$2-fraction of losses are informative, so the effective historical sample is $n\tau$3 rather than $n\tau$4. Under $n\tau$5-differential privacy, the privacy-relevant contribution behaves like $n\tau$6, and the excess risk decomposes into ordinary tail-risk statistical error and a privacy price. In the scalar problem the minimax rate is

$n\tau$7

which makes the dual reduction in usable historical information explicit: first by tail selection, then by privacy (Mansouri, 15 May 2026).

5. Dependence-adjusted effective sample size in structured data

When the historical sample is dependent, EHSS becomes the effective number of independent units represented by a correlated dataset. In phylogenetic comparative methods, this problem is formalized through the covariance matrix induced by a tree and an evolutionary model. Three notions are distinguished: mean ESS, regression ESS, and mutual-information ESS. The regression ESS,

$n\tau$8

is defined from the sum of conditional residual variances after regressing each observation on the others, and is intended to capture observation-level independent signal rather than information about a single mean parameter. In Brownian motion on a highly structured tree, the effective sample size can be close to one; under stronger Ornstein–Uhlenbeck adaptation, dependence decays and ESS approaches $n\tau$9 (Bartoszek, 2015).

For functional spatial data, the dependence summary is the trace-covariogram

$\varepsilon n\tau$0

and the functional ESS is

$\varepsilon n\tau$1

This reduces to $\varepsilon n\tau$2 under independence and to $\varepsilon n\tau$3 under perfect correlation. In a FAR(1) process it becomes a weighted harmonic mean of scalar ESS values across eigen-directions, so both persistence and allocation of variance across functional components determine the effective historical size. In the meteorological application with 600 vertical-velocity curves, fitted models gave ESS values of about 41.9, 102.2, and 105.4, indicating substantial spatial redundancy (Alegría et al., 28 Jan 2026).

A patient-specific version appears in clinical prediction. For a new covariate profile $\varepsilon n\tau$4, the prediction ESS is defined by equating the variance of the model-based prediction to the variance of the sample mean in $\varepsilon n\tau$5 hypothetical patients with the same predictors. In a general GLM,

$\varepsilon n\tau$6

In the GUSTO examples, some patients in the development and external samples had $\varepsilon n\tau$7, especially in the more complex 15-predictor model, indicating that not all predictions were effectively supported by many similar historical patients (Thomassen et al., 2023).

6. Planning, extrapolation, and methodological cautions

EHSS also enters sample size planning through learning curves. For prediction models, expected performance at training size $\varepsilon n\tau$8 is modeled by a deterministic skeleton

$\varepsilon n\tau$9

or by a Gaussian process centered on that curve, with covariance

$\widehat{\mathrm{ESS}}=1/\sum \bar w_n^2$0

The Gaussian-process version borrows information across sample sizes and can be anchored to external datasets by using the posterior for $\widehat{\mathrm{ESS}}=1/\sum \bar w_n^2$1 from the historical dataset as the prior for the target dataset. In the applications, this historical anchoring produced much more stable long-range extrapolation than deterministic curve fitting, and the recommendation is that anchoring against historical evidence when extrapolating sample sizes should be adopted when such data are available (Dayimu et al., 2023).

At the same time, the literature contains important warnings against treating all ESS formulas as interchangeable. In importance sampling, the classical

$\widehat{\mathrm{ESS}}=1/\sum \bar w_n^2$2

is shown to be a heuristic based on multiple approximations. It depends only on normalized weights, is bounded between $\widehat{\mathrm{ESS}}=1/\sum \bar w_n^2$3 and $\widehat{\mathrm{ESS}}=1/\sum \bar w_n^2$4, ignores the integrand $\widehat{\mathrm{ESS}}=1/\sum \bar w_n^2$5, and ignores the sample locations. As a result, it can fail to represent true variance reduction, cannot exceed $\widehat{\mathrm{ESS}}=1/\sum \bar w_n^2$6, and may be unreliable in multiple-importance-sampling or rare-event settings (Tang et al., 2018).

A broader methodological caution follows from comparing the preceding frameworks. ESS can be positive, zero, or negative; scalar or parameterwise; fixed or adaptive; likelihood-free only in appearance; and global or local to an edge, a patient, or a region of the state space. The p-value-based method makes negativity explicit under prior–likelihood discordance, while private CVaR learning shows that privacy can reduce the effective size of a historical dataset even when the nominal sample size is unchanged (Wang et al., 22 Jul 2025, Mansouri, 15 May 2026). This suggests that EHSS should be reported together with its operational definition, the inferential target it is tied to, and the mechanism—borrowing, weighting, dependence adjustment, privacy, or prediction uncertainty—through which nominal data are converted into effective information.