Stratified Sampling: Theory & Applications

• Stratified sampling is defined as partitioning a population into disjoint, homogeneous strata and estimating outcomes separately to achieve lower variance.
• It optimally allocates sample sizes based on stratum variances, outperforming simple random sampling in variance reduction.
• Recent advances integrate quantization, adaptive partitioning, and high-dimensional techniques to improve efficiency and robustness in various applications.

Stratified sampling is a variance reduction technique in which a population or domain is partitioned into disjoint, homogeneous subgroups called strata, and sampling or estimator construction is performed separately (and often optimally) within each stratum. By exploiting knowledge of the heterogeneity structure in the target space or population, stratified sampling achieves lower estimator variance than simple random sampling and is widely used in Monte Carlo methods, experimental design, machine learning, uncertainty quantification, and statistical survey theory. Recent research integrates stratified sampling with quantization, adaptive partitioning, variance-optimal allocation, high-dimensional modeling, federated learning, and streaming data contexts, yielding powerful new methodologies for scientific computation and real-world inference.

1. Principles of Stratified Sampling

Stratified sampling begins by partitioning a domain or dataset into strata $S_1, S_2, ..., S_K$, where each stratum is expected to be more homogeneous with respect to the outcome or response than the full space. The key idea is to separately sample from, or construct estimators within, each stratum and then aggregate the results according to the stratum weights. Let $X$ denote a random variable on $E$ with distribution $\mathbb{P}$, and let $C_i$ be a measurable partition. For a function $F$, the decomposition

$$\mathbb{E}[F(X)] = \sum_{i=1}^K p_i \, \mathbb{E}[F(X) \mid X \in C_i],$$

where $p_i = \mathbb{P}(X \in C_i)$, forms the mathematical basis. Sampling can then target each $C_i$ separately, often with a budget $n_i$ allocated per stratum.

The variance of the stratified estimator (when samples are allocated proportionally, $n_i \propto p_i$) is

$$\operatorname{Var}(\bar{F}_\text{S}) = \frac{1}{N} \sum_{i} p_i\, \sigma_i^2,$$

with $\sigma_i^2$ being the within-stratum variance. This contrasts with simple random sampling, $$\operatorname{Var}(\bar{F}_\text{SRS}) = \operatorname{Var}(F(X))/N$$, and typically yields a strict reduction when the $C_i$ are meaningfully homogeneous.

2. Stratum Design and Quantization

The design of strata significantly impacts the variance reduction. One foundational insight (Corlay et al., 2010) is the use of optimal (quadratic) quantizers for state space partitioning. An $L^2$-optimal quantizer minimizes

$$\min_{\widehat{X}:~\operatorname{card}(\widehat{X})\le N} \| X - \widehat{X} \|_2$$

yielding a Voronoi partition $\{C_i\}$, with centroids $y_i = \mathbb{E}[X|X \in C_i]$. This design is stationary in the quadratic sense and ensures uniform efficiency for the class of Lipschitz functionals:

$\sigma_{F, i} \leq [F]_\mathrm{Lip} \, \sigma_i,$

where $[F]_\mathrm{Lip}$ is the Lipschitz constant. The overall variance reduction is then governed by the quantization error $\| X - \operatorname{Proj}(X) \|_2^2$, providing a notionally universal approach to stratified variance reduction for complex (even infinite-dimensional) distributions.

For Gaussian and functional settings, product quantization using the Karhunen–Loève expansion enables stratification in mode coefficient space, yielding efficient, high-dimensional strata for processes such as Brownian motion or Ornstein–Uhlenbeck (Corlay et al., 2010).

3. Strata Allocation and Variance-Optimal Sampling

Determining the sample size $n_i$ for each stratum is critical. Neyman allocation assigns

$n_i = N \frac{p_i \sigma_i}{\sum_j p_j\sigma_j},$

minimizing the estimator variance under the assumption that all strata are sufficiently abundant. However, in practice, some strata may be bounded (i.e., too small for $n_i$ to be honored), which calls for generalized (variance-optimal) allocations (Nguyen et al., 2018). The VOILA algorithm solves

$$\min_{n_i}~ \sum_i \frac{p_i^2 \sigma_i^2}{n_i},~\text{subject to}~\sum n_i = N,~0 \le n_i \leq N_i,$$

where $N_i$ is the number of available points in stratum $i$.

In streaming contexts, S-VOILA dynamically maintains near-optimal allocations and adapts to changing data distributions while preserving per-stratum randomness. For hybrid or adaptive settings involving unknown or variable $\sigma_i$, convex combinations of proportional and variance-optimal rules are used (Pettersson et al., 2021):

$$N_S^\alpha = p_S N (1 + \alpha (\overline{\sigma}_S - 1)),\qquad \overline{\sigma}_S = \frac{\sigma_S}{\sum_{T \in \mathcal{S}} p_T \sigma_T}.$$

4. Adaptivity, Refinement, and High-Dimensional Strategies

Adaptive stratification refines the strata sequentially in response to the observed variance structure. Algorithms such as Refined Stratified Sampling (RSS) (Shields et al., 2015) select the highest-weighted stratum and split it (typically along the axis of maximal extent), recalculating weights with each extension. Theoretical analysis shows that stratification with balanced or optimal subdivisions strictly reduces the estimator variance.

For network reliability and discrete spaces, unbalanced stratified refinement is essential (Chan et al., 1 Jun 2025). Here, strata are refined according to clusters of components, and only strata with at least $i^*$ failures (the minimum required for system failure) are enumerated, which both concentrates the sampling budget and reduces variance. Heuristic or approximate optimal allocations are used when exact conditional probabilities are computationally infeasible.

High-dimensional stratified sampling is enabled by nonlinear dimensionality reduction (Geraci et al., 10 Jun 2025). Techniques such as Neural Active Manifold (NeurAM) autoencoding collapse input variations onto a 1D latent variable; stratification is then performed in this latent space and mapped back to the input domain. This approach allows scalable application of stratification to problems with many input variables, overcoming the curse of dimensionality and enabling variance reduction in practical multifidelity contexts.

5. Domain-Specific Applications and Extensions

Stratified sampling has found recent domain-specific applications across scientific and engineering contexts:

• Functional and Path-dependent Monte Carlo: Product quantization-based stratification for processes such as Brownian motion, Brownian bridge, and Ornstein–Uhlenbeck (Corlay et al., 2010).
• Graph and Network Sampling: Stratified weighted random walks for graph crawling adapt allocations using category volumes and tailored edge conflict resolution to maximize estimation efficiency in measuring rare or structurally significant subpopulations (Kurant et al., 2011).
• Markov Chains Simulation: Stratified sampling and sorting steps enhance the simulation precision of Markov chains relevant for option pricing and rare event simulation (Fakhereddine et al., 2016).
• Uncertainty Quantification and Stochastic Simulation: Adaptive stratification is used in UQ to target high-variance regions or discontinuities, providing large speedups over standard Monte Carlo sampling (Pettersson et al., 2021).
• Machine Learning and SGD: Stratification in minibatch SGD leverages within-cluster data homogeneity to reduce gradient estimator variance and accelerate convergence (Zhao et al., 2014).
• Experiments and A/B Testing: Subset selection algorithms identify covariates for stratification that maximize variance reduction and statistical sensitivity in online controlled experiments (Momozu et al., 19 Sep 2025).
• Federated Learning: Balanced label exposure and privacy-preserving stratum selection mechanisms combat non-IID data distributions, reduce gradient bias/variance, and accelerate convergence (Wong et al., 18 Apr 2025).

6. Extensions: Distributional Robustness, Dependent Inputs, and High-Fidelity Integration

Distributionally robust extensions address uncertainties in input distributions by optimizing sample allocation over worst-case distributions within specified ambiguity sets (e.g., $L_2$, Wasserstein, moment-based) (Baik et al., 2023). The bi-level optimization framework ensures that the estimator variance is minimized even under input model uncertainty, often using Bayesian optimization for tractability.

When dependencies among input variables preclude marginal stratification, copula or conditional CDF–based transformations (e.g., LHSD) are applied to construct stratified samples that respect the full joint structure, with proven variance reduction over random or naive stratified samples (Mondal et al., 2019).

Other enhancements include the integration of control variates with stratification, as in composite control-variate stratified sampling for efficient molecular integral evaluation (Bayne et al., 2018), and the use of stratified sampling in advanced estimators such as multilevel Monte Carlo (sMLMC) for variance-reduced distribution estimation, leveraging kernel smoothing for further computational efficiency (Taverniers et al., 2019).

7. Practical Considerations and Limitations

The implementation of stratified sampling requires careful stratum design, estimation or approximation of within-stratum variances, and computationally practical allocation strategies. For high-dimensional or complex domains, adaptive or dimension-reducing stratification is essential. In streaming or federated contexts, privacy, dynamic allocation, and heterogeneous participation must be addressed, for instance via secure encrypted protocols or incremental stratification.

While stratified sampling universally reduces variance compared to simple random sampling, the degree of efficiency depends critically on the ability to construct informative and cost-effective strata. In certain extreme non-smooth or combinatorial settings (e.g., network reliability), aggressive pruning and refinement are required. For problems with highly nonlinear, unknown, or latent variable structures, recent advances in nonlinear dimensionality reduction and adaptive refinement are necessary for tractability and efficacy.

---

Stratified sampling thus provides a unified framework for variance reduction and efficient estimation, with theory and methodology extending from classical survey statistics to modern high-dimensional, dynamical, and distributed data analysis scenarios. The ongoing development of adaptive, scalable, and robust stratification schemes continues to expand its impact across computational and inferential domains.