---
title: 'Smoothed Proposals: Efficient Inference'
url: https://www.emergentmind.com/topics/smoothed-proposals
type: topic
---

# Smoothed Proposals: Efficient Inference

Smoothed proposals are central to modern Monte Carlo-based smoothing methods for state-space and diffusion models. In the context of smoothing, the aim is to efficiently construct high-quality proposals for latent trajectory inference, leveraging all available observational data—both past and future. Smoothed proposals extend and improve over purely filtered proposals by incorporating backward information through backward kernels, guided backward simulation, or joint proposal-target construction, with the effect of reducing Monte Carlo variance, mitigating path degeneracy, and improving precision per computation cost. Major algorithmic instances include the Metropolised Forward Filtering Backward Sampling (FFBS) framework, Particle Smoothing Variational Objectives (SVO), and guided proposal schemes in continuous-diffusion smoothing, as well as their Rao–Blackwellised and hybrid extensions [1011.2153] [1909.09734] [1712.03807].

## 1. Theoretical Foundations of Smoothed Proposals

In hidden Markov models and diffusion processes, the smoothing distribution $\pi(x_{0:T}\mid y_{1:T})$ conditions a latent trajectory on all observations. Computation of this distribution is generally intractable outside restricted models (e.g., linear-Gaussian). Monte Carlo techniques, particularly particle methods, approximate the score or finite-dimensional marginals of this smoothing law.

Smoothed proposals are distinguished by their use of smoothing kernels, which, in contrast to filtered (forward) proposals, integrate future as well as past observations. In particle methods, this is achieved via backward kernels derived from the filtering distributions,
$$
L_t(x_{t-1} \mid x_t) = \frac{f(x_t\mid x_{t-1})\;\pi_{t-1}(x_{t-1}\mid y_{1:t-1})}{\int f(x_t\mid u)\;\pi_{t-1}(u\mid y_{1:t-1})\,du}
$$
where $f$ is the transition and $\pi_{t-1}$ the filtered density up to $t-1$ [1011.2153]. These kernels permit the construction of backward trajectories, augmenting the proposal with future-informed dependencies and thus enabling genuine smoothing.

## 2. Metropolised Particle Smoothing and Rao–Blackwellisation

The Metropolised FFBS scheme constructs a Markov chain Monte Carlo (MCMC) kernel using two stages:
- **Forward particle filtering**: simulate and weight $N$ particles via a user-specified proposal $q_t$ and ancestors selected by normalized filtering weights;
- **Smoothed backward proposal**: simulate backward trajectories via the empirical backward kernel
$$
\ell_t^{(j)}(x_t^{(i)}) = \frac{W_{t-1}^{(j)} f(x_t^{(i)} \mid x_{t-1}^{(j)})}{\sum_{k=1}^N W_{t-1}^{(k)} f(x_t^{(i)} \mid x_{t-1}^{(k)})}
$$
sampling at each $t$ from the set of particle ancestors [1011.2153].

This proposal, embedded within a Metropolis–Hastings framework, yields an independent sampler whose acceptance ratio depends only on the marginal likelihood estimates produced by the forward filter,
$$
\alpha = 1 \wedge \frac{\hat Z_T^*}{\hat Z_T}
$$
eliminating the need to compute full path densities recomputed backward.

Rao–Blackwellisation in this context refers to averaging over all possible backward trajectories (“backward smoothing”) rather than sampling a single path, yielding reduced Monte Carlo variance for estimators at computational cost linear in $O(NT)$ [1011.2153].

## 3. Smoothed Proposals in Variational Inference: Particle Smoothing Variational Objectives

Particle Smoothing Variational Objectives (SVO) extend the variational family to smoothed posteriors by augmenting the standard SMC proposal. SVO runs the forward SMC to generate $K$ trajectories, then in the backward pass generates $M$ subparticles per backward time step per trajectory from a continuous proposal $q_\psi$, with categorical subsampling to select the backward trajectory. This constructs a smoothed proposal
$$
q_{\mathrm{SVO}}(\hat{z}_{1:T}^k | x_{1:T}) = \prod_{t=1}^T \Omega_{t}^k
$$
with augmented support (via the $M^T$ subparticles per trajectory) [1909.09734].

The SVO framework further introduces unbiased and biased gradient estimators with respect to inference-network parameters, demonstrating that so-called “drop-the-resampling-gradients” (biased) estimators, or relaxed Concrete/Gumbel-Softmax surrogates, possess signal-to-noise ratio (SNR) scaling $O(\sqrt{K})$ with the number of samples $K$, in contrast to the $O(1/\sqrt{K})$ scaling for unbiased estimators [1909.09734].

Empirical results demonstrate SVO proposals—by informing the proposal with all data—produce more accurate ELBOs and tighter variational bounds in nonlinear latent-dynamics models for fixed computational cost, relative to filtered (forward-only) objectives.

## 4. Guided Smoothing Proposals in Continuous-Discrete Diffusion Models

In continuous-discrete SDE smoothing, smoothed proposals can be constructed via guided bridges. The Backward Filtering Forward Guiding (BFFG) algorithm uses an auxiliary linear Gaussian process to perform backward filtering, yielding tractable time-varying information functions which parameterize a “guided” forward proposal:
$$
dX_t^\circ = \left[ b(X_t^\circ) + a(X_t^\circ)\nabla\log\tilde\rho(t, X_t^\circ) \right]dt + \sigma(X_t^\circ)dW_t
$$
where $a = \sigma\sigma^\top$ and $\tilde\rho$ is the backward information from the auxiliary model [1712.03807].

This forward-guided process serves as a proposal for the true smoothing law, with exactness maintained via Radon–Nikodym derivatives in an MCMC framework. The proposal integrates the backward information from all observations, resulting in efficient sampling and reduced path degeneracy in high-dimensional or non-linear settings [1712.03807].

## 5. Comparative Analysis and Computational Considerations

Smoothed proposals, implemented through FFBS–based or guided approaches, systematically outperform proposal mechanisms that use only forward filtering—especially for tasks requiring statistics of the full path or marginal posteriors significantly before the final observation.

Algorithmic comparison:

| Method                        | Backward Proposal Type   | Complexity per Iteration | Rao–Blackwellisation |
|-------------------------------|-------------------------|-------------------------|----------------------|
| Metropolised FFBS [1011.2153] | Empirical kernel from SMC| $O(NT)$                 | Yes                  |
| SVO [1909.09734]              | Backward continuous + subsampling | $O(KTM)$          | Not direct (averaging over sampled subparticles) |
| BFFG [1712.03807]             | Guided diffusion bridge  | $O(N_{\text{grid}}d^2)$ | Not direct           |

Both Metropolised particle smoothers and SVO exhibit increased precision per CPU second as the proposal becomes more smoothed (i.e., as more backward information or smoothing trajectories are included). Backward smoothing mitigates particle path degeneracy; full Rao–Blackwellisation further reduces Monte Carlo variance at no asymptotic extra cost [1011.2153].

In continuous-discrete diffusion, the BFFG algorithm’s use of smoothed guided proposals achieves exactness (in the continuous-time limit), and demonstrates computational efficiency over local-imputation or disconnected-bridge alternatives [1712.03807].

## 6. Empirical Performance, Applications, and Limitations

Empirical studies in nonlinear latent-dynamics, including FitzHugh–Nagumo, Lorenz attractor, and neuronal voltage trace modeling, demonstrate that smoothed objectives (SVO) achieve tighter predictive $R^2$ and faster ELBO convergence relative to filtered bounds at smaller particle budgets [1909.09734]. In state-space MCMC, FFBS-based proposals produce higher effective sample size per computation especially when genealogical degeneracy undermines filtered-only backward simulations [1011.2153]. For continuous-diffusion smoothing, guided smoothed proposals enable efficient path and parameter inference in chaotic, high-dimensional, and partially observable domains [1712.03807].

The principal limitation of smoothed proposals is computational, as more sophisticated backward passes and smoothing kernels generally scale linearly with both the number of particles and time-steps ($O(NT)$), or, for guided bridges in SDEs, with the number of ODE steps per backward pass.

## 7. Connections to Related Work

Smoothed proposals unify and extend a lineage of particle Markov chain Monte Carlo (PMCMC), SMC, and variational inference approaches. The Andrieu–Doucet–Holenstein (ADH) “Particle Independent Metropolis-Hastings” (PIMH) uses only filtered genealogical-backward sampling and thus is more susceptible to path degeneracy compared to FFBS-based smoothers. SVO builds on FIVO and AESMC, extending the variational proposal’s support by smoothing, which is fundamentally distinct from the time-factorized forward-only construction [1011.2153][1909.09734].

In continuous-time models, guided proposals link to Doob-transformed diffusions and Kalman-based information filtering, extending prior work to fully non-linear, non-Gaussian, and high-dimensional cases [1712.03807].

Smoothed proposals thus form a central methodological pillar in inference for state-space and diffusion models, combining theoretical rigor, practical variance reduction, and adaptability to challenging real-world systems.

Source: https://www.emergentmind.com/topics/smoothed-proposals