Approximate Control Variates (ACV)

• Approximate Control Variates (ACV) are unbiased estimators that reduce variance by correcting high-fidelity outputs with differences from low-fidelity models whose means are estimated rather than known.
• ACV exploits cross-model covariance through multifidelity sampling, enabling integration of expensive simulations with cheaper, correlated evaluations for enhanced efficiency.
• Grouped and parameterized forms of ACV unify various variance-reduction methods and optimize estimator weights, demonstrating gains in applications from numerical integration to deep learning gradient prediction.

Approximate control variates (ACV) are unbiased variance-reduction estimators that extend classical control variates to the practically important setting in which auxiliary-model means are not known exactly and must themselves be estimated. In the canonical multifidelity formulation, a costly high-fidelity estimator is corrected by weighted differences of cheaper, correlated estimators constructed from shared, nested, or otherwise related sample sets. This viewpoint generalizes classical control variates, recovers several multilevel and multifidelity Monte Carlo schemes as special cases, and has since been extended to grouped estimators, distribution estimation, ratio-of-means problems, importance sampling, rendering, attention mechanisms, and gradient prediction (Gorodetsky et al., 2018, Gorodetsky et al., 2024).

1. Formal definition and estimator structure

The classical control-variate estimator assumes that the control means are known:

$$\hat{Q}^{CV} = \hat{Q}+\sum_{\ell=1}^L \alpha_\ell(\hat{Q}_\ell-\mu_\ell),$$

where $\hat Q$ is an unbiased estimator of the high-fidelity quantity $Q$, $\hat Q_\ell$ are low-fidelity estimators, and $\mu_\ell=\mathbb E[Q_\ell]$ are known means. ACV removes the requirement that $\mu_\ell$ be known exactly by replacing each control mean with another estimator. In the notation used in the grouped ACV analysis,

$$\hat{Q}^{ACV}(\alpha_1,\dots,\alpha_L) = \hat{Q}+\sum_{\ell=1}^L \alpha_\ell\left(\hat{Q}_\ell^e-\hat{Q}_\ell^\mu\right),$$

where $\hat Q_\ell^e$ is a correlated mean estimator (CME), $\hat Q_\ell^\mu$ is an approximate mean estimator, and $\Delta_\ell=\hat Q_\ell^e-\hat Q_\ell^\mu$ (Gorodetsky et al., 2024).

A convenient reformulation is

$\hat Q$0

In multifidelity Monte Carlo, the same idea appears as a difference of two low-fidelity estimators evaluated on different sample sets:

$\hat Q$1

Unbiasedness is preserved so long as each component estimator is unbiased, because the correction term has mean zero even though the control mean is itself only estimated (Gorodetsky et al., 2018).

The defining mechanism is covariance exploitation rather than mean replacement alone. ACV chooses the weights $\hat Q$2 so that the low-fidelity terms cancel noise in the baseline estimator while preserving unbiasedness. A notable feature of the modern formulation is that the component estimators $\hat Q$3 need not be standard Monte Carlo averages; they may be any unbiased estimators, which is one reason ACV can absorb a wide range of later constructions (Gorodetsky et al., 2024).

2. Multifidelity foundations and the non-recursive framework

The 2018 multifidelity formulation introduced ACV as a general framework for estimating $\hat Q$4 when a high-fidelity model $\hat Q$5 is paired with cheaper correlated models $\hat Q$6 whose expectations are unknown. In this setting, classical control variates are often unavailable because the low-fidelity means are not known, while naive Monte Carlo ignores exploitable cross-model covariance. ACV addresses this by explicitly accounting for the cost of estimating the control means from additional low-fidelity samples and by optimizing the control weights under a fixed budget (Gorodetsky et al., 2018).

A central theoretical result is that several pre-existing multifidelity estimators are special cases of ACV. In particular, MFMC appears when the sample sets are nested recursively, and recursive difference or MLMC-type estimators arise as ACV schemes with fixed or implicitly recursive weights. The framework therefore unifies classical control variates, MFMC, and recursive-difference estimators in a common linear representation (Gorodetsky et al., 2018).

The same paper also establishes a structural limitation of recursive sampling strategies. For MFMC and weighted recursive difference estimators, the attainable variance reduction is bounded by the best single-control scheme:

$\hat Q$7

The reason given is that recursive constructions induce restricted covariance structures—effectively diagonal for MFMC and tridiagonal for recursive difference—so they cannot recover the full covariance information among all low-fidelity models. The paper describes this as a “single low-fidelity model barrier” (Gorodetsky et al., 2018).

To overcome that barrier, non-recursive ACV sampling strategies were introduced. Two principal families are ACV-IS, which shares the high-fidelity samples with every control while making the extra low-fidelity samples independent across controls, and ACV-MF, which mimics multifidelity nesting but breaks the recursive dependence by using only the first $\hat Q$8 samples in the estimated control-variate mean. Their key convergence property is

$\hat Q$9

so they converge to the variance reduction of the full optimal control variate using all models (Gorodetsky et al., 2018).

This non-recursive viewpoint is especially important when high-fidelity sampling cannot be increased, for example because the expensive simulations come from a legacy database, an experiment, or a previously completed campaign. In that regime, ACV is designed to exploit additional low-fidelity information without requiring more high-fidelity evaluations, and the gap between the best single-control estimator and the full multi-control optimum can be substantial (Gorodetsky et al., 2018).

3. Grouped ACV and the generalized linear estimator viewpoint

The grouped approximate control variate (GACV) formulation extends ACV by treating its components as weighted estimators assembled into groups of models that share input samples. If group $Q$0 contains a subset of models $Q$1, the estimator is written as

$Q$2

This grouping is intended to exploit the covariance induced by sample sharing more effectively than older independent-group constructions (Gorodetsky et al., 2024).

The paper’s main claim is that GACV is a generalized linear estimator: any model evaluation, at any sample, can be assigned an arbitrary weight. When each group estimator is a single model evaluation, GACV can represent any linear combination of model outputs. It is therefore characterized as “the most general linear ansatz for any control-variate type variance reduction scheme,” and it allows each $Q$3 to be not only a Monte Carlo average but also an importance sampling estimator, a surrogate-based estimator, or any other unbiased weighted estimator (Gorodetsky et al., 2024).

Estimator/view	Structural restriction	Status within GACV
ML-BLUE	Independent groups; Monte Carlo group estimators	Special case
Ensemble ACV of Pham et al.	Identical model sets in each group; independent groups; equal high-fidelity group weights; identical per-model weights across groups	Special case
GACV	Arbitrary groups; arbitrary unbiased weighted estimators; groups may share samples	General formulation

A major consequence is that ML-BLUE becomes a literal ACV instance rather than merely a related estimator. The grouped paper proves that any unbiased ML-BLUE can be rewritten as

$Q$4

with the weights obtained by splitting positive and negative coefficients into the two ACV components. This recasts ML-BLUE as a variance-reduction construction rather than only a regression-based one (Gorodetsky et al., 2024).

The grouped formulation also removes the requirement that groups be independent. For independent groups, the covariance matrix is block diagonal and the optimal weights have the closed form

$Q$5

When groups share samples, the covariance matrix acquires off-diagonal blocks. Empirically, the paper shows that non-independent grouped estimators can outperform independent ML-BLUE-style estimators; in many sampled cases, nested non-independent GACV achieves lower variance than the corresponding ML-BLUE, sometimes by a factor of two or more. This suggests that sample reuse across groups is not merely a convenience but can itself be a variance-reduction mechanism (Gorodetsky et al., 2024).

4. Parameterized estimator design and uncertainty in learned weights

Once ACV is viewed as a family of estimators indexed by sample-allocation structure, the central design problem becomes combinatorial. The 2020 optimization study derives a general variance expression for arbitrary ACV sample allocations:

$Q$6

where $Q$7 is the covariance matrix among low-fidelity models, $Q$8 is the covariance vector with the high-fidelity model, and the allocation-dependent factors $Q$9 and $\hat Q_\ell$0 encode overlaps among sample sets. This decomposition makes it possible to optimize directly over much broader families of ACV estimators than MFMC, MLMC, or hand-tailored ACV schemes (Bomarito et al., 2020).

That paper introduces parametrically defined estimator families through a recursion mapping $\hat Q_\ell$1 or $\hat Q_\ell$2 in the independent-sample variant. The resulting families—generalized multifidelity (GMF), generalized recursive difference (GRD), and generalized independent samples (GIS)—contain MFMC, WRDIFF, ACVMF, and ACVIS as special cases. The broader search domain enabled by these parameterizations consistently yields greater variance reduction, and the advantage becomes more pronounced as the number of models increases. With automatic model selection, GMFMR+AMS is reported as dominant in more than 80% of random scenarios (Bomarito et al., 2020).

A separate practical issue is that optimal ACV weights are rarely known in advance; they must be estimated from pilot data or batch ensembles. The ensemble ACV work addresses this by estimating the control weights from $\hat Q_\ell$3 batches and quantifying the additional variance induced by weight uncertainty. Under multivariate Gaussian assumptions for the batch estimators, the variance ratio takes the form

$\hat Q_\ell$4

for $\hat Q_\ell$5 under the paper’s stated sampling assumptions (Pham et al., 2021).

The same analysis gives a sufficient condition for guaranteed variance reduction:

$\hat Q_\ell$6

where the bound depends on the number of controls, the correlation strength, and the sample-allocation ratios. The practical message is that weight-estimation uncertainty can be incorporated analytically rather than treated as a negligible afterthought. The paper also embeds ACV into multi-fidelity importance sampling and reports up to 50% improvement in variance reduction over the existing MFIS estimator on several computational mechanics problems (Pham et al., 2021).

5. Distributional, ratio, and surrogate-based ACV estimators

Although ACV was introduced for expectation estimation, later work extends it to non-scalar targets and nonstandard estimators. For vector-valued CDF estimation, the cvMDL framework constructs a surrogate control variate $\hat Q_\ell$7 from a selected subset of low-fidelity models and uses an approximate control-variate estimator of the form

$\hat Q_\ell$8

The method performs a short exploration phase to identify the most cost-effective model subset, then spends the remaining budget exploiting that subset. Under the paper’s finite-moment, invertibility, admissible-weight, and local Lipschitz assumptions, the resulting estimator is uniformly consistent and asymptotically optimal as the budget tends to infinity, while requiring neither a prescribed hierarchy nor prior knowledge of cross-model statistics (Han et al., 2023).

Ratio-of-means estimation requires an additional modification because both numerator and denominator fluctuate. The 2025 ratio paper introduces control variates for both terms and chooses the coefficients jointly to minimize the variance of the full ratio rather than minimizing the two means separately. In the approximate setting, where the control means are replaced by additional low-cost estimates, the estimator becomes

$\hat Q_\ell$9

A notable theoretical point is that the same jointly optimal coefficients remain optimal in the approximate setting, with the variance reduction scaled by $\mu_\ell=\mathbb E[Q_\ell]$0. In a multi-fidelity aircraft-design use case, the proposed ACV/CV estimator achieves about 20% relative variance reduction for both tested high-fidelity sample sizes (Bocquet-Nouaille et al., 15 Oct 2025).

In rendering and numerical integration, ACV also appears as a surrogate-plus-residual construction. Primary-space adaptive control variates build a piecewise-polynomial approximation that is analytically integrable and use Monte Carlo only for the residual. The method combines quadrature and Monte Carlo in primary sample space, supports multiple importance sampling, and is extended to higher-dimensional integrands by constructing the control variate on a low-dimensional outer variable while estimating the inner high-dimensional component by Monte Carlo. The reported behavior is faster convergence than pure Monte Carlo, pure quadrature, and methods with fixed control variates in several low-dimensional rendering applications (Crespo et al., 2020).

A related regression-based development combines adaptive importance sampling with control variates through weighted least squares rather than through the canonical multifidelity ACV construction. The resulting AISCV estimator can be written as a quadrature rule with adapted weights that do not depend on the integrand, the target density need only be known up to a multiplicative constant, and the paper proves a non-asymptotic probabilistic error bound. This is not presented as the multifidelity ACV framework itself, but it is explicitly described as being in the same spirit: approximate control-variate coefficients are learned from weighted samples produced by adaptive proposals (Leluc et al., 2022).

6. Cross-domain reinterpretations, scope, and recurring principles

Outside uncertainty quantification, the control-variate pattern has been adapted to approximation problems in deep learning. In efficient attention, random-feature-based attention (RFA) is reinterpreted as a sum of local control-variate estimators over tokens. The paper shows that exact softmax attention is recovered if each token receives its own variance-minimizing coefficient, whereas standard RFA corresponds to a single shared coefficient. EVA, the proposed method, partitions the sequence into important tokens handled exactly and remaining tokens handled by group-specific coefficients, while maintaining linear-time and linear-space complexity in sequence length (Zheng et al., 2023).

Gradient prediction offers a second example. The method splits each minibatch into a control micro-batch and a prediction micro-batch, computes true and predicted gradients on the control part, predicted gradients only on the rest, and forms the unbiased update

$\mu_\ell=\mathbb E[Q_\ell]$1

The paper states that the backward pass is typically “2–3 times more expensive than the forward pass,” and the reported CIFAR-10 Vision Transformer experiment uses true gradients on 1/4 of the batch and predicted gradients on 3/4, achieving better validation accuracy throughout a fixed 7200-second training budget than the full-backward baseline (Ciosek et al., 7 Nov 2025).

An earlier variational Bayes line of work provides a related precursor. There, the stochastic gradient of the KL objective is written as

$\mu_\ell=\mathbb E[Q_\ell]$2

and control variates are built from discrepancies between sample covariance terms and exact analytic covariance terms. Under the idealized assumption that the variational approximation and the target posterior share an exponential-family form, the ideal control-variate coefficients reduce to $\mu_\ell=\mathbb E[Q_\ell]$3 for every component, and the estimator becomes variance-free. The paper also identifies a close equivalence between these control variates and stochastic linear regression (Salimans et al., 2014).

Across these settings, a recurring point is that “approximate” does not mean biased. In the multifidelity formulation, the grouped formulation, and the gradient-prediction construction, the approximate or predicted quantity is paired with an unbiased correction term so that the overall estimator retains the correct expectation. A second recurring point is that ACV is not confined to standard Monte Carlo averages, independent groups, or strict model hierarchies: later work explicitly allows arbitrary unbiased weighted estimators, non-independent sample-sharing patterns, and adaptive subset selection without a prescribed hierarchy (Gorodetsky et al., 2024, Gorodetsky et al., 2018). This suggests that ACV is best understood not as a single estimator, but as a broad design principle for building unbiased low-variance estimators from imperfect but correlated auxiliary information.