Multi-Fidelity Stratified Sampling

• Multi-fidelity stratified sampling is a technique that partitions the sampling space by leveraging models of unequal cost and accuracy to reduce predictive uncertainty.
• It integrates adaptive surrogate modeling, stratified Monte Carlo, and control-variate methods to optimize the allocation between low- and high-fidelity evaluations.
• Practical applications range from uncertainty quantification and density estimation to rare-event analysis, offering significant cost savings over high-fidelity-only approaches.

to=arxiv_search бызшәа 天天中彩票充值_code խնդիրը? {"query":"Multi-Fidelity Stratified Sampling arXiv adaptive sampling Gaussian process multi-fidelity stratified", "max_results": 10} Multi-fidelity stratified sampling denotes a family of sampling and allocation procedures that combine models of unequal cost and accuracy with an explicit partitioning principle in order to improve efficiency. In the literature represented here, the partition may be applied to predictive uncertainty across fidelity levels, to the input probability space through strata, to a one-dimensional latent coordinate that induces strata in a high-dimensional domain, or to low-fidelity outputs that are selectively promoted to expensive high-fidelity evaluation. Across these formulations, the common objective is to reduce predictive uncertainty, estimator variance, or failure-probability error per unit cost by exploiting correlation between low- and high-fidelity information rather than treating all simulations as exchangeable (Ghosh et al., 2019, Geraci et al., 10 Jun 2025, Xu et al., 1 Aug 2025).

1. Scope and conceptual variants

The term covers several distinct but related constructions. In adaptive surrogate modeling, the central idea is to choose both the design point and the simulator fidelity by maximizing uncertainty reduction normalized by simulator cost. In stratified Monte Carlo, the key object is a partition of the probability space into strata, followed by within-stratum sampling and weighted recombination. In multifidelity Monte Carlo, stratification can be combined with control-variate estimators so that both within-stratum variance and local high-/low-fidelity correlation contribute to variance reduction. In density estimation, the analogous design question is which low-fidelity outputs should be promoted to expensive high-fidelity evaluation in order to estimate the high-fidelity density, especially its tails (Ghosh et al., 2019, Geraci et al., 10 Jun 2025, Kim et al., 2024).

Formulation	Stratified object	Main mechanism
Adaptive MF-GP	pair $(x,\text{fidelity})$	maximize uncertainty reduction per unit cost
NeurAM-based sMC / sMFMC	unit interval $[0,1]$ pulled back to $D_s$	reduce within-stratum variance and improve local HF/LF correlations
Tail-focused density estimation	low-fidelity output regions	importance-weighted promotion of selected samples
Failure-probability MFSS	strata $E^k$ defined by a stratification variable	stratum-wise MFMC plus total probability theorem

A recurrent misconception is that multi-fidelity stratified sampling must mean classical stratified sampling over a Cartesian partition of the original input space. The cited work shows a broader usage. One line of work explicitly stratifies by fidelity contribution to predictive uncertainty rather than by geometric cells (Ghosh et al., 2019). Another performs stratification on a learned one-dimensional uniform variable and then induces strata in the original high-dimensional domain (Geraci et al., 10 Jun 2025). A plausible implication is that “stratified” should be understood operationally: it refers to structured allocation across heterogeneous subsets—fidelity levels, latent intervals, output regions, or event-conditioned strata—rather than only to fixed partitions in $\mathbb{R}^d$.

2. Fidelity-stratified adaptive sampling for multi-fidelity Gaussian processes

A direct formulation appears in the Kennedy–O’Hagan multi-fidelity Gaussian-process setting, where the high-fidelity response is modeled as

$y(x)=\eta(x,\theta)+\delta(x)+\epsilon.$

Here $\eta(x,\theta)$ is the low-fidelity simulator model, $\delta(x)$ is a discrepancy GP, and $\epsilon$ is i.i.d. Gaussian noise. The paper emphasizes the two-GP structure—one GP for the simulator $\eta(x)$ and one GP for the discrepancy $[0,1]$0—with low-fidelity data $[0,1]$1 and high-fidelity data $[0,1]$2. The single-fidelity kernel is the squared exponential,

$[0,1]$3

and the predictive variance

$[0,1]$4

is the standard uncertainty measure that drives adaptive sampling (Ghosh et al., 2019).

The key insight is that total predictive uncertainty is not monolithic. It has contributions from the low-fidelity surrogate $[0,1]$5 and the discrepancy surrogate $[0,1]$6. A low-fidelity evaluation primarily reduces uncertainty in $[0,1]$7, whereas a high-fidelity evaluation primarily reduces uncertainty in $[0,1]$8 and improves the high-fidelity prediction. The method therefore partitions uncertainty by fidelity level and compares uncertainty reduction against execution cost $[0,1]$9 and $D_s$0. This is the sense in which the paper uses “multi-fidelity stratified sampling”: the search space is stratified by the pair $D_s$1, with each fidelity carrying its own uncertainty-reduction and cost profile.

Three acquisition rules are compared. The baseline, Max MF-UCR, first chooses a location from the overall MF uncertainty and only afterward selects fidelity using the ratio comparison

$D_s$2

The main contribution, Max IF-UCR, turns this into a single-step joint decision: $D_s$3 The Believer-enhanced version, Max IF-UCR Bel, replaces raw uncertainty with a hypothetical posterior-variance reduction under a temporary low- or high-fidelity sample insertion. The Believer approximation assumes that the MCMC-fitted hyperparameters remain unchanged during candidate evaluation, making the search computationally feasible (Ghosh et al., 2019).

The reported behavior is consistent across the 1-D Forrester function, the 4-D Park function, and a 6-D industrial fluidized-bed thermodynamic model. For cost ratios $D_s$4, RMSE converged similarly across methods, but cost differences increased as the high-/low-fidelity cost ratio increased. On the synthetic and industrial examples, Max IF-UCR was generally best, while Max IF-UCR Bel was close, especially at higher ratios. The results support the specific claim that selecting points by total uncertainty alone can waste budget on expensive high-fidelity evaluations when low-fidelity evaluations remove more uncertainty per unit cost.

3. Stratification in high dimensions and multifidelity Monte Carlo

A different line of work addresses the classical difficulty of stratified sampling in high dimensions. Let $D_s$5 on $D_s$6, $D_s$7, and $D_s$8. Standard Monte Carlo has estimator

$D_s$9

Classical stratification partitions the domain into $E^k$0 and uses

$E^k$1

with variance

$E^k$2

The obstacle is the curse of dimensionality: regular grids in $E^k$3 become combinatorially expensive (Geraci et al., 10 Jun 2025).

The proposed remedy is to stratify a one-dimensional uniform variable instead of the original input space. Using NeurAM, a nonlinear dimensionality-reduction method, the paper learns an encoder $E^k$4, a decoder $E^k$5, and a 1D surrogate $E^k$6. If $E^k$7 and $E^k$8 is the CDF of $E^k$9, then

$\mathbb{R}^d$0

With intervals $\mathbb{R}^d$1, the induced strata in the original space are

$\mathbb{R}^d$2

A key lemma gives $\mathbb{R}^d$3, so stratum probabilities are known exactly from interval lengths. Sampling within a stratum can be obtained by rejection on the latent uniform coordinate.

This construction supports both ordinary stratified Monte Carlo and stratified multifidelity Monte Carlo. For sMC, the paper analyzes optimal allocation

$\mathbb{R}^d$4

and proportional allocation $\mathbb{R}^d$5, proving

$\mathbb{R}^d$6

For MFMC, the standard control-variate estimator uses an expensive $\mathbb{R}^d$7, a cheap $\mathbb{R}^d$8, and cost ratio $\mathbb{R}^d$9. The stratified version applies MFMC separately in each stratum with local correlation $y(x)=\eta(x,\theta)+\delta(x)+\epsilon.$0 and local coefficient $y(x)=\eta(x,\theta)+\delta(x)+\epsilon.$1. The paper gives sufficient conditions under which stratified MFMC improves on standard MFMC, notably when stratification raises local high-/low-fidelity correlations. Empirically, NeurAM-based stratification beats standard Monte Carlo consistently, often beats classical grid-based stratification, remains effective up to 10 dimensions, and further improves multifidelity Monte Carlo estimators (Geraci et al., 10 Jun 2025).

4. Control-variate covariance structure and sample-overlap design

A complementary perspective treats multi-fidelity stratification as a design problem over sample overlaps. In the approximate control-variate framework, one has a high-fidelity estimator $y(x)=\eta(x,\theta)+\delta(x)+\epsilon.$2, $y(x)=\eta(x,\theta)+\delta(x)+\epsilon.$3 low-fidelity estimators $y(x)=\eta(x,\theta)+\delta(x)+\epsilon.$4, and discrepancy vector

$y(x)=\eta(x,\theta)+\delta(x)+\epsilon.$5

The practical ACV estimator replaces the unknown expectations by sample-based approximations: $y(x)=\eta(x,\theta)+\delta(x)+\epsilon.$6 Its variance is minimized by

$y(x)=\eta(x,\theta)+\delta(x)+\epsilon.$7

yielding the standard explained-correlation reduction term (Dixon et al., 2023).

The distinctive contribution is the explicit use of overlapping sample sets. For two sample sets $y(x)=\eta(x,\theta)+\delta(x)+\epsilon.$8 and $y(x)=\eta(x,\theta)+\delta(x)+\epsilon.$9, the overlap count is

$\eta(x,\theta)$0

For multi-output mean estimators,

$\eta(x,\theta)$1

where $\eta(x,\theta)$2. Thus the covariance between estimators is directly controlled by shared samples. The paper extends this logic to variance estimators, stacked mean–variance estimators, and Sobol main-effect variance estimators, introducing the higher-order covariance blocks needed for optimal ACV or MLBLUE-like designs. MLMC and MFMC appear as special cases with particular nested sample structures, whereas ACV is more general. This suggests that, in a broad statistical sense, multi-fidelity stratified sampling includes the deliberate arrangement of overlap patterns so that low-fidelity evaluations become maximally informative under a budget constraint.

The practical consequence is that allocation is not determined by correlation alone. Sample-sharing structure and model cost both enter the design objective. The paper formulates an explicit budget-constrained optimization of the estimator covariance determinant, and its numerical studies show that multi-output ACV can outperform separate single-output ACV estimators by more than an order of magnitude in variance reduction when output correlations are strong (Dixon et al., 2023).

5. Importance-sampled promotion of low-fidelity outputs for density and tail estimation

When the target is not an expectation but the high-fidelity density $\eta(x,\theta)$3, the relevant sampling question becomes which low-fidelity outputs should be promoted to high-fidelity evaluation. The setup uses paired outputs $\eta(x,\theta)$4, where $\eta(x,\theta)$5 is cheap low fidelity, $\eta(x,\theta)$6 is expensive high fidelity, and both are driven by the same underlying random seed. The paper considers both homoscedastic and heteroscedastic relations,

$\eta(x,\theta)$7

and studies density estimation of $\eta(x,\theta)$8, with special attention to the tails (Kim et al., 2024).

The proposed method samples low-fidelity outputs according to a proposal density $\eta(x,\theta)$9, evaluates high fidelity only on the selected points, and then uses importance-weighted kernel density estimation,

$\delta(x)$0

The $\delta(x)$1-domain is divided into a left tail, a central region, and a right tail. The tails retain the actual $\delta(x)$2 behavior, while the center uses a user-chosen $\delta(x)$3. The variance expression

$\delta(x)$4

serves as the optimality criterion.

In the noiseless monotone case $\delta(x)$5, the derived optimal proposal in the central region is

$\delta(x)$6

so the design allocates more expensive evaluations where the low-to-high map changes rapidly. For piecewise monotone $\delta(x)$7, the optimal proposal becomes

$\delta(x)$8

favoring low-fidelity values whose mapped high-fidelity outputs lie in low-density, hence more informative, regions of the target distribution. In the heteroscedastic case, the method recommends a Box–Cox transform $\delta(x)$9 and then applies the same logic on the transformed scale.

Tail treatment is separated from central density estimation. The central region uses the weighted kernel estimator, while the far tails are modeled using extreme value theory and generalized Pareto distributions motivated by the Pickands–Balkema–De Haan theorem. The practical selection rule is explicit: preserve all extreme low-fidelity values, sample the center according to an importance proposal, and then evaluate high fidelity only on those selected points. In simulation studies and a ship-motions application comparing SimpleCode and LAMP, the modified estimator improves tail estimation relative to the plain kernel estimator, especially beyond the observed data range (Kim et al., 2024).

6. Rare-event failure analysis with adaptive machine-learning MFSS

A full multi-fidelity stratified-sampling framework for rare-event analysis appears in nonlinear structural dynamics. The system response is

$\epsilon$0

with scalar quantity of interest $\epsilon$1, and failure probability

$\epsilon$2

through the total probability theorem. The probability space is partitioned into mutually exclusive and exhaustive strata $\epsilon$3, defined by a stratification variable $\epsilon$4 that is cheap to evaluate and highly correlated with the system response. In the case study, $\epsilon$5, the elastic resultant base moment (Xu et al., 1 Aug 2025).

The method combines generalized stratified sampling, MFMC, and an adaptive deep-learning metamodel that serves as the low-fidelity model. A very large Phase-I Monte Carlo sample estimates stratum probabilities $\epsilon$6 and defines the strata. In the reported study, the space was partitioned into $\epsilon$7 strata using $\epsilon$8 Phase-I MC samples, with the smallest-probability stratum satisfying

$\epsilon$9

The low-fidelity model is a GRU-based sequence-to-sequence surrogate in a reduced POD space, with one GRU layer, 200 hidden units, dropout probability $\eta(x)$0, Adam optimizer, learning rate $\eta(x)$1, MSE loss, wavelet decomposition level $\eta(x)$2, and POD truncation $\eta(x)$3, yielding $\eta(x)$4 reduced modes from $\eta(x)$5.

Adaptive training is driven by the reduced-space weighted correlation coefficient

$\eta(x)$6

estimated by K-fold cross-validation. Training stops when

$\eta(x)$7

In the case study, $\eta(x)$8, $\eta(x)$9, and the achieved values were $[0,1]$00 and $[0,1]$01. The paper reports that 130 total training samples were sufficient and that further additions produced only marginal improvement.

Within each stratum, the conditional failure probability is estimated by an MFMC correction using high-fidelity and low-fidelity evaluations not reused from the metamodel training set, preserving unbiasedness. The global estimator is

$[0,1]$02

For the building example, the cost ratio was

$[0,1]$03

and the final sampling budget used $[0,1]$04 and $[0,1]$05 per stratum, amounting to 110 HF evaluations and 39,880 LF evaluations, compared with 1,500 HF evaluations for an HF-only GSS approach. Reported comparisons for four limit states show lower COV for MFSS than GSS, and the overall speedup was

$[0,1]$06

with computational budget about 16% of the HF-only GSS budget. The paper also notes that curves using only the GRU low-fidelity model are biased, whereas MFSS corrects this bias with a small high-fidelity correction set (Xu et al., 1 Aug 2025).

7. Related surrogate-allocation methods and conceptual boundaries

Not every multi-fidelity sampling method that allocates many cheap evaluations and few expensive evaluations is a classical stratified sampler. A representative example is the multi-fidelity neural-network surrogate sampling method for uncertainty quantification of ODE/PDE systems. It begins with low- and high-fidelity models $[0,1]$07 and $[0,1]$08, assumes a general nonlinear relation

$[0,1]$09

and trains two neural networks: NN1 learns the low-/high-fidelity correlation from paired data, and NN2 is trained on an augmented dataset in which NN1 supplies approximate high-fidelity labels at additional low-fidelity points. The resulting surrogate $[0,1]$10 is then embedded in Monte Carlo: $[0,1]$11 The paper explicitly states that this is not classical stratified sampling in the variance-reduction sense, but it is a two-level allocation of cheap and expensive model evaluations that plays a similar cost-saving role (Motamed, 2019).

Its error decomposition,

$[0,1]$12

splits deterministic surrogate error from Monte Carlo sampling error and allocates a target tolerance $[0,1]$13 between them using a parameter $[0,1]$14. The asymptotic cost comparison is central: standard high-fidelity Monte Carlo behaves like

$[0,1]$15

whereas, if training costs are asymptotically negligible and NN evaluation is cheap,

$[0,1]$16

In the reported ODE and PDE examples, the method approaches $[0,1]$17, while classical Monte Carlo scales as $[0,1]$18 and $[0,1]$19, respectively.

This boundary case is important because it clarifies the breadth of the field. Multi-fidelity stratified sampling sometimes denotes exact stratum-wise unbiased estimators with total-probability aggregation or local control variates; sometimes it denotes fidelity-aware uncertainty partition in adaptive design; and sometimes it denotes surrogate-based two-level allocation that is “related in spirit” but not identical to classical stratification. The literature therefore does not support a single universal definition. What it does support is a consistent design principle: exploit unequal costs and cross-fidelity dependence so that expensive simulations are spent only where they provide information that low-fidelity evaluations cannot provide at comparable cost.