--- title: Smooth Monte-Carlo Estimators url: https://www.emergentmind.com/topics/smooth-monte-carlo-estimators type: topic --- # Smooth Monte-Carlo Estimators Smooth Monte-Carlo estimators are Monte Carlo procedures in which smoothness is not incidental but part of the estimator design. In the literature represented here, smoothness enters in several distinct ways: by smoothing noisy batch means before robust aggregation, by fitting smooth surrogate functions whose integrals are analytically tractable, by replacing i.i.d. sampling with repulsive or lattice-based point sets adapted to smooth integrands, by smoothing discontinuous simulation outputs through conditioning or kernels, by stabilizing noisy autocovariance sequences in MCMC, and by using randomized Taylor truncations to estimate smooth functions of expectations without bias [1411.4074], [1603.03220], [2604.19698], [1604.06008], [1807.06133], [1906.04607], [1706.00853], [2403.20313], [2211.07422]. ## 1. Conceptual scope A useful way to organize this area is by asking what is being smoothed. In some constructions, the estimator itself is smoothed: the “smooth” median-of-means of randomized approximation schemes multiplies each batch mean by an independent \(R \sim \mathrm{Unif}([1-\epsilon,1+\epsilon])\), so that outliers are softened before median aggregation [1411.4074]. In others, the target integrand is approximated by a smooth control functional or regression surrogate, and Monte Carlo is applied only to the residual [1603.03220], [2211.07422]. A third class smooths the sampling design rather than the function, using structured point sets such as projection DPPs or randomly shifted and dilated lattices to exploit spectral or Sobolev regularity [2604.19698], [1604.06008]. A fourth class smooths discontinuous outputs or diagnostics, for example by conditioning to turn indicator-based density estimators into continuous conditional densities, or by imposing PSD and monotonicity constraints on empirical autocovariance sequences in MCMC [1906.04607], [2103.15976], [1706.00853]. This literature also suggests that “smoothness” has two mathematically different roles. In one role, smoothness is an assumption on \(f\), \(\pi\), or \(\widehat f\), and sharper convergence follows because Fourier tails, RKHS approximation error, or polynomial coefficients decay rapidly. In the other role, smoothness is a device for variance control even when only weak moments are available, as in relative-error estimation under \(\mathrm{SD}(X)\le c\mu\), or in conditional Monte Carlo constructions that replace discontinuous indicators by continuous conditional expectations [1411.4074], [1603.03220], [1604.06008], [1906.04607]. ## 2. Smoothed relative-error estimators in randomized approximation schemes A central problem in randomized approximation schemes is to estimate a strictly positive mean \(\mu\) from i.i.d. samples \(X_1,X_2,\ldots\) under the coefficient-of-variation bound \(\sigma \le c\mu\), with guarantee \[ \mathbb P(|\hat\mu-\mu|>\epsilon\mu)\le \delta, \] for \(\epsilon\in(0,1/3)\) and \(\delta\in(0,1)\). The construction in “Improving Monte Carlo randomized approximation schemes” replaces the classic block-median argument by a smoothed batch statistic. For batch size \[ m=\left\lceil (c/\epsilon)^2 f(\epsilon)\right\rceil,\qquad f(\epsilon)=(1-\epsilon)^{-2}(1+\epsilon-\epsilon^2)^{-1}(1+\epsilon), \] one forms batch means \(S_j\), draws \(R_j\sim \mathrm{Unif}([1-\epsilon,1+\epsilon])\) independently, sets \(V_j=S_jR_j\), and returns the median of \(V_1,\dots,V_b\) over \[ b=2\left\lceil \frac{\ln(2\delta^{-1})}{\ln(4/3)}\right\rceil+1 \] batches [1411.4074]. The key smoothing lemma states that if \(S\) has mean \(\mu\) and standard deviation at most \((\epsilon/\sqrt{f(\epsilon)})\mu\), then with independent \(R\sim \mathrm{Unif}([1-\epsilon,1+\epsilon])\), \[ \mathbb P(SR\ge \mu+\epsilon\mu)\le \tfrac14,\qquad \mathbb P(SR\le \mu-\epsilon\mu)\le \tfrac14. \] This per-batch tail probability \(p=1/4\) is then propagated through a median concentration bound for \(2k+1\) independent batches, giving \[ \mathbb P(|\hat\mu-\mu|>\epsilon\mu)\le \delta \] with total sample size \[ n=\left\lceil (c/\epsilon)^2 f(\epsilon)\right\rceil \left[2\left\lceil \frac{\ln(2\delta^{-1})}{\ln(4/3)}\right\rceil+1\right]. \] In leading order, \[ n=6.96\,(c/\epsilon)^2\ln(\delta^{-1})+o\!\big((c/\epsilon)^2\ln(\delta^{-1})\big), \] improving on the classic median-of-means constant \(19.35\) [1411.4074]. The significance of this result is not only the reduced constant. The smoothing step yields the two-sided \(1/4\) batch failure probability without the larger block sizes that Chebyshev-based arguments would require. The paper explicitly situates the method in randomized approximation schemes for the volume of a convex body, the permanent of a nonnegative matrix, the number of linear extensions of a poset, and partition functions of the Ising model. The guarantee depends critically on \(\mu>0\), independence of samples, and correct specification of the bound \(\sigma\le c\mu\); if \(\mu\le 0\), if \(\mu\) is extremely small, or if the samples are dependent, the relative-error formulation becomes ill-posed or the concentration argument can fail [1411.4074]. ## 3. Control functionals and regression surrogates A major line of work smooths Monte Carlo by replacing a rough integrand with a learned surrogate whose integral is known. In “Convergence Rates for a Class of Estimators Based on Stein’s Method”, the surrogate has the form \[ f_m(x)=\beta+S_\pi[\phi](x), \qquad S_\pi[\phi](x)=\nabla\!\cdot\phi(x)+\phi(x)\cdot \nabla \log \pi(x), \] with \(\phi\in \mathcal H^d\) in an RKHS and \(\int S_\pi[\phi]\,d\Pi=0\). Under sample splitting, \(D_0\) is used to fit \(f_m\) by minimum-norm interpolation in the augmented RKHS \(\mathcal H_+=\mathcal H_{\mathbb R}\oplus \mathcal H_0\), and \(D_1\) is used to compute \[ I_{m,n}=\frac{1}{n-m}\sum_{i=m+1}^n \Big[f(x_i)-\big(f_m(x_i)-\textstyle\int f_m\,d\Pi\big)\Big]. \] The estimator is unbiased, and its conditional variance is \((n-m)^{-1}\sigma^2(f-f_m)\) [1603.03220]. The principal rate theorem is a scattered-data approximation statement. If \(\pi\) has smoothness \(a\), the kernel has smoothness \(b\), the domain satisfies an interior cone condition, and \(f\in \mathcal H_+\), then in the independent case \[ \mathbb E_{D_0}\mathbb E_{D_1}\!\left[ 1_{h_{D_0}0\). The same MSE rate extends to uniformly ergodic reversible Markov chains via variance-bounding arguments. At the same time, the gain deteriorates with dimension through the factor \(m^{-(a\wedge b)/d}\), which the paper identifies as an inherent curse of dimensionality [1603.03220]. A related but computationally simpler approach appears in “Regression-based Monte Carlo Integration”. There, Monte Carlo integration is reinterpreted as estimation of a constant function, and then generalized to an analytically integrable surrogate \(g\). Writing \(G=\int g\) and \[ \hat F=G+\frac1N\sum_{i=1}^N \big[f(u_i)-g(u_i)\big], \] one obtains an unbiased control-variate estimator whose variance is \((1/N)\mathrm{Var}(f-g)\). When \(g\) is fitted by least squares over a basis that includes the constant function, the resulting estimator is provably better than or equal to conventional Monte Carlo, because the constant model is always available as a fallback. In the rendering experiments reported in the paper, low-order polynomial surrogates acted as a practical drop-in replacement for standard Monte Carlo and combined effectively with MIS [2211.07422]. Taken together, these papers suggest a common structural principle: learn a smooth or low-complexity approximation whose integral is available in closed form, then use stochastic sampling only for the residual. The distinction is that Stein control functionals obtain the zero-mean correction from \(\nabla \log \pi\) and RKHS structure, whereas regression-based Monte Carlo relies on basis functions with analytically known expectations [1603.03220], [2211.07422]. ## 4. Structured point sets: determinantal processes and randomized lattices Another family of smooth Monte-Carlo estimators leaves the integrand unchanged and instead regularizes the point set. In “On two ways to use determinantal point processes for Monte Carlo integration”, projection DPPs are used to generate exactly \(N\) repulsive nodes from a kernel \[ K_N(x,y)=\sum_{k=0}^{N-1}\phi_k(x)\phi_k(y). \] The Bardenet–Hardy estimator is \[ \mathrm{BH}_N(f)=\sum_{n=1}^N \frac{f(x_n)}{K_N(x_n,x_n)}, \] which is unbiased. For the multivariate Jacobi ensemble on \([-1,1]^d\) and \(f\) essentially \(C^1\), the cited CLT implies variance of order \(N^{-(1+1/d)}\), hence RMSE \(N^{-(1+1/d)/2}\). The Ermakov–Zolotukhin estimator uses the same DPP nodes but solves a linear system in the basis \(\{\phi_k\}\); when \(\phi_0\) is constant, it yields an unbiased integral estimator whose variance is exactly the residual \(L^2(\mu)\) energy of \(f\) outside the span \(\mathcal H_N=\mathrm{span}\{\phi_0,\dots,\phi_{N-1}\}\). If \(f\in \mathcal H_N\), the variance is zero [2604.19698]. The same paper emphasizes that the two estimators exploit smoothness differently. BH uses a fixed DPP and benefits when the smoothness of \(f\omega\) yields small Fourier coefficients relative to the arcsine equilibrium measure. EZ is tailored to \(f\) through the spectral residual and can integrate finite expansions exactly, but it may be numerically erratic because the interpolation matrix can be ill-conditioned and the resulting weights need not be nonnegative. Exact chain-rule sampling algorithms are provided for multivariate Jacobi ensembles, with expected rejection count approximately \(2^dN\log N\), and a rejection-free \(O(N^2)\) tridiagonal sampler is available in one dimension [2604.19698]. A different structured-sampling mechanism appears in “A Monte Carlo method for integration of multivariate smooth functions”. There the estimator is a randomly shifted and dilated Frolov lattice rule \[ M_n(f)=\frac{1}{\det(UB_n)}\sum_{m\in \mathbb Z^d} f\big((UB_n)^{-\top}(m+v)\big), \] with \(u\sim U([1/2,3/2]^d)\), \(U=\mathrm{diag}(u)\), and \(v\sim U([0,1]^d)\). For compactly supported \(f\in L_2(\mathbb R^d)\), \[ \Delta(M_n,f)\le C_{d,B}\,n^{-1/2}\,\|\widehat f\|_{L_2(D_n)}, \] so the RMS error is the high-frequency \(L_2\)-mass of \(\widehat f\) outside a growing neighborhood of the origin. This immediately converts Fourier decay into Sobolev rates. For mixed smoothness \(H_p^s([0,1]^d)\) with \(p\ge 2\), the optimal randomized order on the cube is \(n^{-s-1/2}\), independent of dimension. For isotropic smoothness, the rate becomes \(n^{-s/d-1/2}\) [1604.06008]. The conceptual contrast is instructive. DPP estimators obtain variance reduction through repulsion and spectral projection, while randomized Frolov rules obtain it through controlled aliasing in Fourier space. Both methods, however, convert smoothness into decay of a spectral tail, and both depart from classical Monte Carlo by imposing strong structure on the point set rather than on the estimator algebra [2604.19698], [1604.06008]. ## 5. Density estimation and conditioning-based smoothers Simulation-based density estimation makes the role of smoothing especially explicit. In the scalar KDE setting \[ \hat f_h(x)=\frac1{nh}\sum_{i=1}^n k\!\left(\frac{x-X_i}{h}\right), \qquad X_i=g(U_i), \] standard Monte Carlo gives \[ \mathrm{IV}_{\mathrm{MC}}(h)=\frac{\mu_0(k^2)}{nh}-\frac1n R(f)+O(h/n), \qquad \mathrm{ISB}(h)=\frac{\mu_2(k)^2}{4}R(f'')\,h^4, \] so \(\mathrm{MISE}_{\mathrm{MC}}=\Theta(n^{-4/5})\) at bandwidth \(h=\Theta(n^{-1/5})\). Replacing MC by RQMC or stratification changes only the variance, not the bias. “Density estimation by Randomized Quasi-Monte Carlo” shows that Koksma–Hlawka-type bounds lead to \[ \mathrm{IV}_{\mathrm{RQMC}}(h)=O\!\big(n^{-2+\epsilon}h^{-2s}\big), \] which is too loose in moderate or high dimension because the Hardy–Krause variation grows like \(h^{-s}\). For nested uniform scrambling, however, \[ \mathrm{IV}_{\mathrm{NUS}}(h)\le 2^t3^s\left(\frac{\mu_0(k^2)}{nh}-\frac{R(f)}n+O(h/n)\right), \] so the optimal bandwidth scaling remains \(n^{-1/5}\) while constants can improve substantially. Under monotone \(g\), stratification yields \[ \mathrm{IV}_{\mathrm{Strat}}(h)\le (b-a)s\,k(0)^2\,h^{-2}n^{-(s+1)/s}, \] with \(\mathrm{MISE}=\Theta(n^{-(2/3)(s+1)/s})\) [1807.06133]. The review “Density Estimation by Monte Carlo and Quasi-Monte Carlo” broadens this picture by comparing KDE with conditional density estimators, smoothed perturbation analysis, LR, and GLR-U constructions. The central identity is \(F(x)=\mathbb E[1\{X\le x\}]\), \(f(x)=F'(x)\). The problem with the empirical cdf is that it is discontinuous in the underlying uniforms and its derivative is a sum of deltas. Conditioning replaces the indicator by a smooth conditional cdf or conditional density. In “Monte Carlo and Quasi-Monte Carlo Density Estimation via Conditioning”, if \(F(x\mid V)\) is continuous in \(x\), differentiable except on a countable set, and \(F'(x\mid V)\) is uniformly bounded by an \(L^2\) random variable \(\Gamma\), then \[ \hat f_n(x)=\frac1n\sum_{i=1}^n f_{X\mid V}(x\mid V_i) \] is unbiased, with \(\mathrm{MISE}\le (b-a)K_\gamma/n\). Under bounded Hardy–Krause variation of the derivative integrand, RQMC improves the pointwise MSE to \(O(n^{-2+\epsilon})\). The paper reports especially large gains for the cantilever beam, stochastic activity network, and Asian option examples, with empirical log–log slopes near \(2\) for several CDE+RQMC configurations [2103.15976], [1906.04607]. At a more general level, “A Monte Carlo Method to Approximate Conditional Expectations based on a Theorem of Besicovitch” justifies shrinking-neighborhood Monte Carlo approximations of conditional expectations by a differentiation-of-measures theorem. For \(U:\Omega\to \mathbb R^n\) and integrable \(f\), the paper shows that for \(P^U\)-almost every \(u\), \[ E(f\mid U=u)= \lim_{\epsilon\downarrow 0}\lim_{k\to\infty} \frac{\sum_{i=1}^k 1\{U(\omega_i)\in B_\epsilon(u)\}f(\omega_i)} {\sum_{i=1}^k 1\{U(\omega_i)\in B_\epsilon(u)\}}, \] providing a nonparametric conditional-expectation mechanism even when densities are unavailable [1306.1182]. Across these density-estimation papers, smoothing serves a precise purpose: it transforms discontinuous, high-variation simulation outputs into continuous functions of the driving uniforms, thereby making RQMC, conditioning, and derivative-based arguments effective [1807.06133], [2103.15976], [1906.04607], [1306.1182]. ## 6. MCMC covariance smoothing and stable Monte Carlo error assessment Not all smooth Monte-Carlo estimators target means or integrals directly. In “Multivariate initial sequence estimators in Markov chain Monte Carlo”, smoothness refers to stabilization of the estimated long-run covariance matrix in the multivariate MCMC CLT \[ \sqrt n(\mu_n-\mu)\Rightarrow \mathcal N_p(0,\Sigma). \] Writing \(\Gamma_i=\gamma_{2i}+\gamma_{2i+1}\) for adjacent-sum autocovariances, Proposition 1 shows that in the population sequence each \(\Gamma_i\) is positive definite, \(\Gamma_i-\Gamma_{i+1}\) is positive definite, and \(\Gamma_i\to 0\). The empirical counterparts \(\Gamma_{n,i}\) are noisy, and naive truncation can lead to unstable or non-PSD covariance estimates [1706.00853]. The multivariate initial-sequence estimator mIS addresses this by enforcing shape constraints. One forms \[ \Sigma_{n,m}=-\gamma_{n,0}+2\sum_{i=0}^m \Gamma_{n,i}, \] lets \(s_n\) be the first index for which \(\Sigma_{n,s_n}\) is positive definite, and then takes \(t_n\) as the last index before the determinant sequence \(|\Sigma_{n,m}|\) ceases to increase. The estimator is \(\Sigma_{\mathrm{seq},n}=\Sigma_{n,t_n}\). The adjusted version mISadj replaces each \(\Gamma_{n,i}\) beyond \(s_n\) by its PSD projection \(\Gamma_{n,i}^+\), ensuring that increments cannot shrink the confidence ellipsoid in negative-eigenvalue directions. Theorems 2 and 3 show that both mIS and mISadj conservatively estimate the generalized variance, in the sense that \[ \liminf_{n\to\infty} |\Sigma_{\mathrm{seq},n}|\ge |\Sigma|, \qquad \liminf_{n\to\infty} |\Sigma_{\mathrm{adj},n}|\ge |\Sigma| \] almost surely [1706.00853]. The practical role of smoothing here is shape-constrained regularization of a noisy time-domain spectral estimate. The paper contrasts mIS and mISadj with Kosorok’s multivariate estimator mK, which truncates at the first PSD violation and can underestimate generalized variance, and with componentwise univariate initial-sequence estimators, which ignore cross-correlation and yield cube-shaped rather than ellipsoidal uncertainty regions. The result is not differentiability but stability: positive-definite covariance estimates, more reliable Monte Carlo standard errors, and confidence ellipsoids with empirically improved coverage [1706.00853]. ## 7. Unbiased nonlinear transforms of expectations and recurrent limitations A distinct use of smoothness appears in “Towards a turnkey approach to unbiased Monte Carlo estimation of smooth functions of expectations”. Here the objective is not \(\mathbb E[X]\) itself but \(f(m)\), where \(m=\mathbb E[X]\) and \(f\) is analytic, with special emphasis on \(f(x)=\log x\) and \(f(x)=1/x\). Expanding around \(c\neq 0\), \[ f(m)=\sum_{k=0}^\infty \gamma_k\big[(m/c)-1\big]^k, \qquad \gamma_k=\frac{f^{(k)}(c)}{k!}c^k, \] one can unbiasedly estimate \(((m/c)-1)^k\) by products of independent terms \((X_i/c)-1\). The paper studies both the simple estimator \(Z_k^S\) and a variance-reduced cycling estimator \(U_{r,k}^C\) that averages cyclic products across all \(r\) samples. With an independent truncation \(N\) and weights \(w_k=1/P(N\ge k)\), the sum estimator \[ \hat f=\sum_{k=0}^N w_k\gamma_k U_{N,k} \] is unbiased for \(f(m)\) [2403.20313]. The finite-variance theory is driven by \[ \beta_0=\left|\frac{m}{c}-1\right|, \qquad \beta^2=\frac{\sigma^2}{c^2}+\beta_0^2. \] For geometric truncation \(N\sim \mathrm{Geometric}(p)\), the random-truncation variance behaves like \(O(p)\), but the sampling-noise term behaves very differently for the two estimators: the simple estimator does not have vanishing noise variance as \(p\to 0\), while the cycling estimator satisfies an \(O(p\log(1/p))\) bound, i.e. near Monte Carlo rate up to a logarithmic factor. The paper proposes automatic tuning via a pilot sample, with \[ c^*=\frac{m^2+\sigma^2}{m}, \qquad p=\min(1-\hat\beta^2,\;1/(n_0+1)), \] and a one-sided bootstrap safeguard for the minimum admissible \(c\) [2403.20313]. This line of work is especially relevant to latent-variable likelihood estimation and unnormalized models, because \(\log m\) and \(1/m\) arise naturally in those settings. Its limitations are equally explicit: for \(\log x\) and \(1/x\), one needs \(m>0\), \(c>0\), \(|(m-c)/c|<1\), and geometric rather than sub-geometric truncation tails; heavy-tailed or high-variance \(X\) can make \(\beta^2\) close to \(1\), forcing very small \(p\) and therefore large expected truncation levels [2403.20313]. Viewed together, these papers reveal recurrent trade-offs. Gains from smoothness often deteriorate with dimension through scattered-data fill distance, polynomial or Fourier residuals, or DPP sampling cost [1603.03220], [2604.19698], [1604.06008]. Some methods require access to \(\nabla\log\pi\), compact domains, or boundary mollifiers [1603.03220]. Others depend on positivity and correctly specified variance surrogates, as in relative-error randomized approximation schemes and unbiased reciprocal or logarithmic transforms [1411.4074], [2403.20313]. In RQMC density estimation, classical Koksma–Hlawka bounds become unusable because variation blows up as \(h^{-s}\), even though empirical performance can still be strong [1807.06133]. In MCMC covariance smoothing, reversibility, a multivariate CLT, and reliable autocovariance estimation remain essential [1706.00853]. The unifying implication is not that there is a single canonical “smooth Monte-Carlo estimator”, but that multiple Monte Carlo subfields repeatedly exploit the same principle: replace a discontinuous, noisy, or spectrally diffuse object by one with more regular structure, then use stochastic sampling on what remains. The specific regularity may be uniform scaling of batch means, RKHS smoothness, polynomial or Fourier sparsity, repulsive node geometry, conditional differentiability, PSD shape constraints, or analyticity of \(f\). The estimator class changes, but the operational objective is stable: lower variance, stronger nonasymptotic concentration, or more reliable Monte Carlo error quantification from the same underlying simulation budget [1411.4074], [1603.03220], [2604.19698], [1906.04607], [1706.00853], [2403.20313]. Source: https://www.emergentmind.com/topics/smooth-monte-carlo-estimators