---
title: Recursive Adaptive Importance Sampling
url: https://www.emergentmind.com/topics/recursive-adaptive-importance-sampling
type: topic
---

# Recursive Adaptive Importance Sampling

Searching arXiv for recent and foundational papers on recursive adaptive importance sampling and closely related adaptive importance-sampling frameworks.
Recursive adaptive importance sampling denotes a family of Monte Carlo methods in which the proposal distribution is not fixed in advance but is updated online from previously generated weighted samples. The common objective is to estimate an integral or expectation while simultaneously learning a proposal that reduces importance-weight variability, improves effective sample size, or balances statistical efficiency against computational cost. In the modern literature, the topic encompasses single-sample stochastic-gradient recursions, stagewise adaptive multiple importance sampling with recycling, tempering-based stabilization, adaptive proposal-family fitting, and recursive Bayesian weight updates with replenishment [1412.4845] [1806.00989].

## 1. Formal setting and core definitions

A standard formulation begins with a target quantity
\[
I=\mathbf{E}\,\phi(X)=\int \phi(x)f(x)\;dx,
\]
where \(X\sim f\). Ordinary Monte Carlo draws IID samples from \(f\) and uses
\[
\hat{I}_n^\mathrm{MC}=\frac{1}{n}\sum^n_{i=1}\phi(X_i).
\]
Importance sampling instead introduces a proposal density \(\tilde f\), with \(\tilde f(x)>0\) whenever \(\phi(x)f(x)\neq 0\), and forms
\[
\hat{I}^{\mathrm{IS}}_n=\frac{1}{n}\sum^n_{i=1}\phi(X_i)\frac{f(X_i)}{\tilde{f}(X_i)}.
\]
This estimator is unbiased, and its variance is governed by the second moment of the weighted integrand [1412.4845].

Recursive adaptive importance sampling replaces the fixed proposal by a sequence \(\tilde f_1,\tilde f_2,\ldots\), with \(X_i\sim \tilde f_i\), and estimator
\[
\hat{I}_n^\mathrm{AIS} = \frac{1}{n}\sum^n_{i=1}\phi(X_i)\frac{f(X_i)}{\tilde{f}_i(X_i)}.
\]
The recursive feature is that \(\tilde f_{i+1}\) is selected from information accumulated up to iteration \(i\). In stage-based formulations, stage \(t\) consists of an exploration step using \(n_t\) samples from \(q_{t-1}\), followed by an exploitation step that updates the sampling policy \(q_t\); the resulting empirical sums have a martingale structure, and the adaptive estimator remains centered at the target integral under support conditions [1806.00989].

Across the literature, the target of adaptation varies. Some methods adapt the natural parameter of an exponential-family proposal to minimize variance [1412.4845]. Others adapt a Gaussian-mixture proposal by fitting weighted samples to intermediate targets [2205.01501]. In rare-event stochastic root-finding, the adaptive object is an importance-sampling parameter \(\alpha_{n+1}=I(\hat\theta_n)\) determined by the current root estimate [2102.10631]. In recursive Bayesian computation, weighted particles are propagated through likelihood increments and periodically replenished by a refitted proposal [2509.08102]. A broader interpretation also includes procedures that adapt the number of proposals inside an importance-resampling Markov kernel rather than the proposal density itself [2512.00220].

## 2. Recursive update architectures

The simplest recursive architecture uses each new sample for two tasks simultaneously: estimation of the target quantity and update of the proposal for future sampling. This one-pass coupling is explicit in Convex AdaMC, where each sample contributes one importance weight to the Monte Carlo average and one stochastic gradient step for the next proposal parameter [1412.4845]. A similar structure appears in stochastic approximation for quantile estimation, where the current sample under \(P_{\alpha_n}\) updates \(\hat q_n\), and the new sampler is then set to \(\alpha_{n+1}=I(\hat q_n)\) [2102.10631].

A second architecture is stagewise or batchwise. In the asymptotic-optimality framework, stage \(t\) draws \(n_t\) samples from \(q_{t-1}\), updates the estimator, and then updates \(q_t\); no restriction is imposed on the allocation policy \((n_t)\), so the theory does not require a prescribed split between exploration and exploitation [1806.00989]. Adaptive multiple importance sampling methods follow this pattern as well: a current batch is sampled, weighted, and then used to fit the next proposal, often while retaining all past samples for a final multiple-importance denominator [1211.2548].

A third architecture updates a posterior approximation recursively with incoming data. RAISOR exploits
\[
[\theta \mid y_{1:n+1}] \propto [y_{n+1}\mid \theta, y_{1:n}] [\theta \mid y_{1:n}],
\]
which yields the cheap weight recursion
\[
\tilde w_{k+1}(\theta_m) \propto \tilde w_k(\theta_m)\,[y_{k+1}\mid y_{1:k},\theta_m].
\]
Because repeated weight updates induce degeneracy, replenishment steps are inserted at selected times; the proposal is then adapted to the current weighted sample and a fresh particle cloud is drawn [2509.08102].

The literature also contains a deterministic analogue. Quantization based recursive Importance Sampling keeps the same optimization target as stochastic-approximation RIS but replaces random Monte Carlo estimation of the criterion and its derivatives by finite deterministic sums under an optimal quantizer, followed by Newton–Raphson zero search [1109.4041]. This is explicitly presented as a deterministic counterpart of recursive importance sampling.

## 3. Variance minimization and convex stochastic programming

A central line of work formulates recursive adaptation as an optimization problem over a proposal family. For exponential-family proposals
\[
f_\theta(x)=\exp \left(\theta^TT(x)-A(\theta)\right)h(x),
\]
the per-sample variance of the importance-sampling estimator is
\[
V(\theta)=\int \frac{\phi^2(x)f^2(x)}{f_\theta(x)}\;dx-I^2.
\]
The optimization problem is to minimize \(V(\theta)\) over a convex parameter set \(\Theta\) [1412.4845].

The key structural theorem is that \(V(\theta)\) is a convex function of the natural parameter \(\theta\). The proof uses convexity of the log-partition function \(A(\theta)\), and the variance objective is written as an integral of convex functions of \(\theta\). This convexity is important because it rules out the local-minimum pathologies that often complicate adaptive importance sampling outside exponential-family settings [1412.4845].

Convex AdaMC performs stochastic gradient descent on this variance objective while accumulating the Monte Carlo estimate:
\[
X_n\sim f_{\theta_n},
\]
\[
\hat{I}^\mathrm{AMC}_n = \frac{1}{n}\sum^n_{i=1}\frac{\phi(X_i)f(X_i)}{f_{\theta_i}(X_i)},
\]
\[
g_n=(\nabla A(\theta_n)-T(X_n))\frac{\phi^2(X_n)f^2(X_n)}{f_{\theta_n}^2(X_n)},
\]
\[
\theta_{n+1}=\Pi\left(\theta_n-\frac{C}{\sqrt{n}}g_n\right),
\]
where \(\Pi\) is Euclidean projection onto \(\Theta\). The stochastic gradient is unbiased because
\[
\nabla V(\theta)= E_{X\sim f_\theta} \left[ (\nabla A(\theta)-T(X))\frac{\phi^2(X)f^2(X)}{f_\theta^2(X)} \right].
\]
The resulting estimator is unbiased for \(I\) even though the proposal changes over time [1412.4845].

The same direct-optimization idea also appears in structured probabilistic domains. In Bayesian networks and influence diagrams, the proposal is parameterized by conditional-probability tables, and a sequential stochastic-gradient update
\[
\theta^{(t+1)}=\theta^{(t)}-\alpha^{(t)} \nabla_\theta e(\theta^{(t)})
\]
is applied either to the variance objective itself or to discrepancy measures between the current proposal and approximations of the optimal proposal. In that setting, the minimum-variance proposal for positive \(g\) is
\[
f^*(Z)=\frac{g(Z)}{\sum_Z g(Z)},
\]
which is unavailable because the normalizing constant is the unknown target quantity; recursive adaptation is introduced precisely to approximate this unavailable optimum online [1301.3882].

## 4. Stabilization by recycling, tempering, weighting, and replenishment

Recursive adaptation creates a recurring stability problem: early proposals may be poor, high-variance weights can distort learning, and repeated reweighting can collapse the effective sample size. Several strands of the literature address this issue by modifying the adaptation target rather than the basic importance ratio.

AMIS introduces a recycling denominator that uses the mixture of all proposals visited so far:
\[
\omega_i^k = \frac{\pi(X_i^k)}{\Omega_t^{-1}\sum_{\ell=1}^{t}N_\ell q(X_i^k,\widehat\theta_\ell)}.
\]
The consistency analysis shows that full recycling inside the learning step is delicate; the proved version therefore updates \(\widehat\theta_{t+1}\) using only the current sample with ordinary weights, and performs only one final recycling step after adaptation is complete. The paper’s methodological lesson is that recursive adaptation and recycling can be separated: current samples drive learning, while all samples can still be reused in the final estimator [1211.2548].

TAMIS stabilizes adaptation by introducing a tempered intermediate target
\[
\pi_{\beta,t}(x) \propto \pi(x)^\beta q_t(x)^{1-\beta}, \qquad \beta\in(0,1),
\]
which transforms the weights to \(w_{t,i}^\beta\), and then an anti-truncation map
\[
\widehat w_{t,i}^{\beta}= s \vee w_{t,i}^{\beta} = \max\{s,\,w_{t,i}^{\beta}\}.
\]
Tempering reduces weight degeneracy, while anti-truncation preserves contamination from the current proposal \(q_t\) and helps prevent premature collapse. The tempering parameter is calibrated automatically by an ESS constraint, and the threshold \(s_t\) is taken as a quantile of the tempered weights [2205.01501].

Weighted AIS addresses the finite-sample cost of poor early stages by introducing stage weights
\[
I_T^{(\alpha)}(\psi) = \frac1{N_T} \sum_{t=1}^T \alpha_{T,t} \sum_{i=1}^{n_t} \frac{\psi(x_{t,i})}{q_{t-1}(x_{t,i})},
\]
with \(\sum_{t=1}^T n_t \alpha_{T,t}=N_T\). The stated purpose is to forget poor samples from early stages while preserving unbiasedness for unnormalized integrals [1806.00989].

RAISOR treats degeneration through replenishment rather than weight transformation. Sample quality is monitored by the Relative Effective Sample Size
\[
\text{RESS}(n\mid n_0) = \left\{1+D_{\chi^2}\!\left([\theta\mid y_{1:n}] \,\|\, [\theta\mid y_{1:n_0}]\right)\right\}^{-1},
\]
with empirical estimator
\[
\widehat{\text{RESS}}(n\mid n_0) = \frac{\left\{\frac{1}{M}\sum_{m=1}^M w(\theta_m)\right\}^2} {\frac{1}{M}\sum_{m=1}^M w^2(\theta_m)}.
\]
When estimated RESS falls below a threshold, the method refits the proposal and regenerates particles [2509.08102].

## 5. Proposal families and adaptive quantities

The umbrella term covers markedly different choices of proposal family and adaptive variable.

| Method | Proposal or kernel | Adaptive quantity |
|---|---|---|
| Convex AdaMC | Exponential family \(f_\theta\) | Natural parameter \(\theta\) |
| Modified AMIS | Parametric proposal \(Q(\widehat\theta_t)\) | Proposal parameter \(\widehat\theta_t\) |
| TAMIS | Gaussian mixture model with diagonal covariances | \(\beta_t\), \(s_t\), and mixture parameters |
| DAIS | Gaussian \(q_t=\mathcal N(\mu_t,\Gamma_t)\) | \(\mu_t\), \(\Gamma_t\), and damping \(\epsilon_t\) |
| VAE-based AIS | VAE proposal density | Neural-network parameters of the proposal |
| Adaptive i-SIR | i-SIR transition \(P_\lambda\) | Proposal count \(\lambda\) |
| RAISOR | Weighted particles plus replenishment proposal | Replenishment times and proposal parameter \(\eta^{(s)}\) |

Within Gaussian approximation methods, DAIS forms the damped target
\[
q_{t,\epsilon}(x)\propto q_t^{\,1-\epsilon}(x)\,\pi(x)^{\epsilon},
\]
or equivalently
\[
q_{t,\epsilon}(x)\propto q_t(x)\exp\{\epsilon\,\Phi_t(x)\},\qquad \Phi_t(x)=\log\pi(x)-\log q_t(x),
\]
and then updates the mean and covariance of the Gaussian approximation through Stein-identity-based moment corrections. The damping parameter \(\epsilon_t\) is chosen as the largest value for which the current effective sample size exceeds a fixed threshold, so the method adapts both the proposal and the amount of tempering [2404.18556].

High-dimensional non-parametric AIS replaces Gaussian or Gaussian-mixture proposals by a variational autoencoder. The proposal density is learned from weighted samples using a weighted ELBO, together with a learnable prior based on the VampPrior and a pre-training procedure intended to reduce posterior collapse. The learned proposal is then inserted into existing adaptive importance-sampling algorithms and cross-entropy rare-event procedures [2310.09194].

Adaptive i-SIR occupies a boundary position. It is not a general proposal-adaptation scheme, since it does not adapt \(q\); instead, it introduces a fractional proposal count \(\lambda\ge 1\), a generalized kernel
\[
P_\lambda(x,A)=\beta P_N(x,A)+(1-\beta)P_{N+1}(x,A),
\]
and a stochastic-approximation update of \(\lambda_k = 1+e^{\xi_k}\) based on an approximate efficiency criterion. It is therefore recursive and importance-sampling-based, but its adaptive variable is computational effort rather than the proposal law [2512.00220].

## 6. Asymptotic guarantees, applications, and limitations

A defining feature of the subject is that many adaptive procedures come with asymptotic guarantees showing that online learning does not destroy first-order efficiency. In the stagewise AIS framework, if the proposal sequence converges suitably and a moment condition holds, then
\[
\sqrt n \left(I_n-\int\varphi\right)\xrightarrow{d}\mathcal N(0,V_*),
\]
and in parametric AIS, consistency of the sampling policy yields the oracle property
\[
\sqrt n\,(I_n-I)\xrightarrow{d}\mathcal N\big(0,V(q_{\theta_*},\varphi)\big),
\]
meaning that adaptive learning is asymptotically as efficient as using the limiting optimal proposal from the outset [1806.00989].

Convex AdaMC strengthens this theme in a single-sample recursive setting. Under compactness of \(\Theta\) and a finite fourth-moment condition, the estimator variance satisfies
\[
\frac{1}{n}V^\star\le \operatorname{Var}(\hat{I}^\mathrm{AMC}_n)\le \frac{1}{n}V^\star+\left(\frac{D^2}{2C}+CG^2\right)\frac{1}{n^{3/2}},
\]
so
\[
\operatorname{Var}(\hat{I}^\mathrm{AMC}_n)=\frac{1}{n}V^\star+\mathcal{O}(n^{-3/2}),
\]
and
\[
\sqrt{n}(\hat{I}^\mathrm{AMC}_n-I) \stackrel{\mathcal{D}}{\rightarrow} \mathcal{N}(0,V^\star).
\]
The limiting variance is the best achievable within the chosen exponential family [1412.4845].

In stochastic root-finding and quantile estimation, adaptive importance sampling resolves the circular dependence between the unknown solution and the unknown optimal sampler. The resulting SAA, Robbins–Monro SA, and Polyak–Ruppert SA estimators are strongly consistent and asymptotically normal, and the asymptotic variance matches that of the fixed optimal sampler at the true solution. Numerical examples report large variance reductions for extreme quantiles, including standard normal, exponential, Pareto-tailed, and portfolio VaR/CVaR settings [2102.10631].

Applications span several domains. Structured-domain methods target belief inference in Bayesian networks and action evaluation in influence diagrams [1301.3882]. Quantization-based recursive IS is developed for option pricing in both multi-dimensional and path-dependent diffusion settings [1109.4041]. TAMIS is evaluated on banana-shaped targets and high-dimensional Gaussian targets [2205.01501]. DAIS targets Gaussian posterior approximation for intractable Bayesian posteriors [2404.18556]. VAE-based AIS addresses multimodal sampling and rare-event estimation in high dimensions [2310.09194]. RAISOR is applied to Gaussian-process regression for sea surface temperature prediction in the Gulf of Mexico [2509.08102].

Several limitations recur across the literature. Proposal-family misspecification remains fundamental: if the proposal family cannot represent the target well, KL divergence may not vanish, the temperature may not reach \(1\), or the algorithm may converge only to the best available approximation inside the chosen family [2205.01501]. Convex-programming guarantees for Convex AdaMC require an exponential-family proposal, a convex compact parameter set, and finite fourth moments, and the constants in the variance bound are “not very informative for tuning in practice” [1412.4845]. AMIS consistency is established only for a modified learning rule, not for the fully recycled learning dynamics of the original algorithm [1211.2548]. RAISOR is explicitly described as less attractive in high-dimensional settings, where degeneration accelerates rapidly [2509.08102]. DAIS is recursive and adaptive, but it is not standard SMC because it does not maintain a resampled particle cloud across iterations [2404.18556]. Adaptive i-SIR provides a theory for tuning the number of proposals, not for adapting the proposal distribution itself [2512.00220].

A common misconception is therefore that “recursive adaptive importance sampling” names a single algorithmic template. The literature instead shows a family resemblance: sequential proposal improvement from weighted samples, preservation or recovery of asymptotic exactness, and explicit mechanisms for controlling instability caused by changing proposals. What differs is the adaptive variable—proposal parameter, temporary target, damping level, recycling weight, replenishment schedule, or proposal count—and the mathematical tool used to justify it.

Source: https://www.emergentmind.com/topics/recursive-adaptive-importance-sampling