---
title: Sample Extensions in Statistical Inference
url: https://www.emergentmind.com/topics/sample-extensions
type: topic
---

# Sample Extensions in Statistical Inference

Sample extensions are methodological constructions that enlarge a procedure beyond the specific sample, sample size, or sample space for which it was originally defined. In the literature considered here, the phrase covers several technically distinct operations: out-of-sample placement of unseen observations in a learned representation, enlargement of a sampling mechanism to a richer or continuously weighted state space, extension of sample-selection and sensitivity-analysis procedures to larger and more irregular datasets, and translation of prior or design information into observation-equivalent units. The common idea is to reuse structure already extracted from data—an embedding, a kernel, a partition, a coupling relaxation, or a prior/posterior summary—so that new inference can be performed without re-solving the full original problem [1802.06307] [2408.04129] [1406.5958] [2504.14673].

## 1. Conceptual scope

No single standardized definition of sample extension spans all of these literatures. Instead, the term designates a family of related moves in which a method learned on one sample regime is made operative on another.

| Setting | What is extended | Typical aim |
|---|---|---|
| Representation learning | Embedding, kernel, or projection | Place unseen observations in existing coordinates |
| Combinatorial sampling | Sampling state space or weighted family | Sample constrained objects or count them |
| Large-scale analysis | Selection or sensitivity estimator | Reduce memory and runtime burdens |
| Statistical planning | Prior or design information | Express impact in units of observations |
| Distributional modeling | Finite-sample or discrete construction | Lift to continuous measures or transformed models |

Representative examples include adjacency spectral embedding and positive-semidefinite-kernel out-of-sample formulas [1802.06307] [1711.07271], out-of-core dimensionality reduction via reference subsets [2408.04129], weighted sampling families for linear extensions [1010.4981] and random-to-random Markov chains on \(\mathcal L(P)\) [1412.7488], entropy-regularized sample selection [2503.19874], and sample-size or effective-sample-size extensions for A/B testing, prior diagnostics, and Gaussian graphical models [2305.16459] [1406.5958] [2606.22687].

## 2. Out-of-sample extension in embeddings and graph inference

In graph representation learning, the sample-extension problem is posed after an adjacency spectral embedding has already been computed. For an \(n\)-vertex graph with adjacency matrix \(A\), the in-sample embedding is \(\hat X=U_A S_A^{1/2}\). A new vertex arrives with edge-indicator vector \(a\in\{0,1\}^n\), and the goal is to place it in the same coordinate system using only \(\hat X\) and \(a\), rather than recomputing the eigendecomposition of the augmented graph. The least-squares extension solves \(\min_w \|\hat X w-a\|^2\) and has closed form \(\hat w_{\mathrm{LS}}=S_A^{-1/2}U_A^\top a\); the companion plug-in maximum-likelihood extension maximizes a constrained Bernoulli log-likelihood over \(\widehat{\mathcal T}_\epsilon\). Under the random dot product graph model, both estimators recover the new latent position up to orthogonal transformation with error \(O(\log n/\sqrt n)\), and the least-squares estimator satisfies a central limit theorem with asymptotic covariance \(\Sigma_w=\Delta^{-1}\mathbb E[X_1^\top w(1-X_1^\top w)X_1X_1^\top]\Delta^{-1}\) [1802.06307].

A different out-of-sample construction appears in nonlinear kernel embedding. There the learned object is not a fixed diffusion kernel but a positive semi-definite kernel \(B_\star\) obtained from a semidefinite program with diagonal constraints. In the weak-smoothness setting, each coordinate is extended by a normalized Nyström formula chosen so that the squared norm of the out-of-sample point equals \(d(x)\), and the learned matrix \(\mathbf B_\star\) extends to a data-dependent Mercer kernel \(b_{\star,e}(x,y)=\Xi_e(x)^\top \Xi_e(y)\). In the strong-smoothness RKHS setting, the extension becomes the kernel expansion \(b(x,y)=\sum_{i,j}k(x,x_i)[R^{-1}BR^{-T}]_{ij}k(x_j,y)\), which preserves positive semi-definiteness automatically [1711.07271].

The same general principle scales standard dimensionality-reduction methods to data volumes that cannot be embedded jointly in memory. A small reference subset \(X_r\) is embedded directly, yielding \(Y_r\), and the remaining data are processed in batches by an out-of-sample transform tied to the chosen method. The paper evaluates MDS, PCA, t-SNE, UMAP, and autoencoders; reports visualization of datasets with up to 50 million data points; and gives a use case involving one billion projected streamline instances. The computational advantage comes from fixing the reference embedding once, but the trade-off is that later samples cannot globally reorganize that geometry [2408.04129].

## 3. Extensions of sampling spaces and weighted sample families

In combinatorial sampling, sample extension concerns the state space of sampled objects rather than unseen observations. For a finite poset \(P\), a linear extension is a total order compatible with the partial order, and the state space is \(\mathcal L(P)\). One line of work extends the classical random-to-random shuffle on permutations to this constrained space. Adjacent operators \(\tau_i\) exchange neighboring elements only when they are incomparable, and the random-to-random move \(T_{i,j}\) is formed by composing such local moves. The resulting \(P\)-random-to-random shuffle has transition matrix \(M_P\); its central conjecture is that the second largest eigenvalue satisfies \(\lambda_2(P)\le (1+1/n)(1-2/n)\), with equality for disconnected posets, implying relaxation time at most \(n^2/(n+2)\) and mixing time \(O(n^2\log n)\). The paper proves the conjectured bound for direct sums of chains and for \(N\)-shaped posets, and proves that \((1+1/n)(1-2/n)\) is an eigenvalue for every disconnected poset [1412.7488].

A second line extends exact-uniform sampling of linear extensions into a continuously parameterized family of weighted distributions. Starting from a fixed “home” extension, the paper defines weights \(w(\sigma,\beta)\) that interpolate continuously between the single home ordering and the full set of linear extensions. This yields nested sets \(A(\beta)=\{(x,y):x\in L,\ y\in[0,w(x,\beta)]\}\) with \(\mu(A(n-1))/\mu(A(0))=\#(L)\), so the Tootsie Pop Algorithm can estimate the number of linear extensions by repeatedly sampling from \(P_\beta(\sigma)\propto w(\sigma,\beta)\). The resulting two-phase procedure returns an estimate within a factor \(1+\epsilon\) of \(\#(L)\) with probability at least \(1-\delta\), and the weighted samples themselves are generated exactly by a modified adjacent-transposition chain together with non-Markovian coupling from the past [1010.4981].

These two papers illustrate distinct notions of extension. One extends a Markov move from permutations to poset-constrained orderings; the other embeds a discrete counting problem into a continuous family of weighted samples so that exact sampling can drive approximation. The shared structure is that neither approach starts from scratch on a new state space: each reuses an existing combinatorial mechanism and enlarges it just enough to retain control of mixing or counting.

## 4. Sample-selection and scalable estimator extensions

Another usage of sample extension appears when the aim is neither embedding nor exact combinatorial sampling, but adaptation of a procedure so that it remains effective on much larger sample sets. In optimal design, the regret-minimization framework is extended by adding an entropy regularizer and by allowing a regularized information matrix. The entropy-regularized objective
\[
\min_{w\in\Delta_n} \ \Tr\!\left(\left(\sum_{i=1}^n w_i x_i x_i^\top\right)^{-1}\right)+\lambda\sum_{i=1}^n w_i\log w_i
\]
produces a new sample-selection criterion that prefers more spread-out weights, while the regularized version replaces \(M(w)\) by \(M_\gamma(w)=\gamma I+\sum_i w_i x_i x_i^\top\). The paper states a provable \((1+\epsilon)\)-near optimal guarantee and applies the method to unlabeled subset selection on MNIST, CIFAR-10, and a 50-class subset of ImageNet, where it consistently outperforms competing sampling strategies in most cases [2503.19874].

For variance-based sensitivity analysis, the extension takes the form of a generalized given-data Sobol’ estimator that is no longer tied to equiprobable bins or all-at-once memory layouts. With an arbitrary partition \(\Omega_{X_i}=\bigcup_{k=1}^M A_k\), the estimator uses
\[
\widehat{EV}=\sum_{k=1}^M s^2(X_i\in A_k)P_k,\qquad P_k=\frac{n_k}{N},
\]
followed by
\[
\hat S_i = 1-\frac{\widehat{EV}}{\widehat V}.
\]
A streaming algorithm updates per-bin counts, means, and unscaled variances batchwise, and a heuristic \(4\sigma\) filter removes indices too small to distinguish from zero under statistical noise. These modifications are designed for models with \(>10^4\) parameterizable inputs, including neural networks, and the paper reports comparable accuracy and runtimes with lower memory requirements than methods that require all samples simultaneously [2509.09078].

A notable point in this literature is that the extension is not purely computational. In the Sobol’ case, the paper argues that equiprobable partitions can introduce substantial bias because the approximation
\[
\mathbb E[\mathbb V(f(\mathbf X)\mid X_i)\mid X_i\in A_k]\approx \mathbb V(f(\mathbf X)\mid X_i\in A_k)
\]
can fail most severely in low-density tail bins, and equiprobable binning weights those bins equally. The extension to arbitrary partitions therefore changes both feasibility and statistical behavior.

## 5. Extensions of sample-size and prior-information accounting

In Bayesian diagnostics and experimental design, sample extension often means turning an abstract influence measure into a quantity expressed in units of observations. One framework defines the prior-information function \(M(I)\) by matching posterior uncertainty under a target prior to posterior uncertainty under a baseline prior:
\[
U_{\pi_b}(I+M(I))=U_\pi(I).
\]
Here \(D\) can be posterior MSE, and \(I\) is a scalar information index, typically the sample size \(k\). The extension is threefold: it works beyond conjugate families, treats prior sample size as a relative function \(M(k)\), and permits \(M(k)<0\) to encode harmful prior–likelihood conflict. In the normal benchmark, the effective prior contribution depends not only on the nominal prior size \(m\) but also on the standardized discordance \(\Delta\), and the slope of \(M(k)\) becomes a graphical diagnostic of prior–likelihood mismatch [1406.5958].

A pre-data analogue for Gaussian graphical models extends effective sample size to Wishart and G-Wishart priors on the precision matrix. The paper adapts five ESS estimators to the GGM setting, aggregates them either globally through determinant ratios or parameterwise through a Cholesky decomposition, and then builds two planning tools on top of them: the Data-to-Prior Information Ratio, which finds the sample size at which the data dominate the prior, and a GGM extension of Bayes Factor Design Analysis, which finds the sample size needed for conclusive edgewise evidence. The central contribution is to express prior informativeness for \(\Theta\) in observation-equivalent units even when graph constraints and matrix dependence make classical pseudo-count interpretations unavailable [2606.22687].

A/B testing produces a further extension of sample-size formulas toward realistic online-experiment regimes. For clustered or repeated-measure settings, the paper replaces the iid variance term \(\sigma^2/n\) by a Delta-method expression \(h/k\), yielding
\[
k = 2h\,(z_{1-\alpha/2}+z_{1-\beta})^2/\delta^2
\]
at the randomization-unit level. It also derives dedicated formulas for relative lift rather than treating percentage effects as simple plug-in rescalings of absolute effects, and it links the design-stage target effect to the smallest observed difference likely to be significant through
\[
|\Delta_{\mathrm{MOD}}| \approx \frac{z_{1-\alpha/2}}{z_{1-\alpha/2}+z_{1-\beta}}\,|\Delta_{\mathrm{ATE}}|.
\]
The paper’s broader point is that design formulas must be extended whenever correlation or ratio estimands are built into the analysis itself [2305.16459].

## 6. Continuous and model-transform extensions over probability distributions

Some of the most abstract sample extensions operate directly on probability measures. For Gromov–Wasserstein, the discrete moment-SOS hierarchy is extended to compact metric measure spaces by replacing finite moment matrices with a measure \(P\in\mathcal P((X\times Y)^{2r})\) satisfying symmetry, marginal consistency, and a continuous positive-semidefiniteness condition expressed through quadratic forms in measurable test functions. The resulting hierarchy \(gw^{(r)}(\mu,\nu)\) is a genuine lower-bounding sequence, converges to the GW objective as \(r\to\infty\), induces a pseudo-metric \(GW_{p,q}^{(r)}\), and remains statistically meaningful under empirical sampling, with expected empirical relaxed distances converging to their population counterparts [2504.14673].

A different distribution-level extension appears in models generated by randomly stopped extremes. If \(N\) is a positive-integer-valued stopping variable with pgf \(h_N\), then the \(N\)-stopped maximum of a parent variable \(X\) has cdf \(h_N(F_X)\), the \(N\)-stopped minimum has cdf \(\bar h_N(F_X)=1-h_N(1-F_X)\), and the paper defines inverse constructions—the \(N\)-maxprecursor and \(N\)-minprecursor—with cdfs \(h_N^{-1}(F_X)\) and \(\bar h_N^{-1}(F_X)\). By composing extreme and precursor maps, one obtains model extensions that always embed the original family. When the stopping model is closed under pgf composition, these extensions are statistically stable in the sense that reapplying them leaves the enlarged family unchanged; when the stopping law is extreme auto-reversible, maxima-based and minima-based extensions coincide. The zero-truncated geometric case recovers the Marshall–Olkin extension [2412.15997].

Taken together, these works suggest that sample extension is less a single method than a recurring design pattern. A structure learned on a manageable sample or under a simpler sampling regime is enlarged—by algebraic inversion, kernel continuation, streaming summary statistics, posterior-risk matching, or measure-theoretic relaxation—so that it continues to operate in a broader domain. What changes across fields is the object being preserved: coordinate alignment, positivity, stationary distribution, pseudo-metric structure, or calibration in observation units. What remains constant is the objective of extending usefulness without paying the full cost of recomputation or redesign.

Source: https://www.emergentmind.com/topics/sample-extensions