Controlled Sequential Monte Carlo

• Controlled Sequential Monte Carlo is a method that improves proposal mechanisms by solving an optimal control problem to achieve near zero-variance estimators.
• It twists path measures via a policy-driven approach and recursive Bellman equations, aligning particle trajectories with the target distribution.
• Approximate dynamic programming and adaptable policy classes enable cSMC to robustly handle both state-space and static models in complex inference settings.

Controlled Sequential Monte Carlo (cSMC) is a Sequential Monte Carlo methodology in which the proposal mechanism is improved by solving, approximately and iteratively, an associated optimal control / dynamic programming problem (Heng et al., 2017). In the modern formulation, cSMC replaces a fixed proposal sequence by a sequence of twisted path measures parameterized by a policy, and seeks a policy that makes the SMC weights as flat as possible—ideally constant, yielding a zero-variance normalizing-constant estimator (Heng et al., 2017). The term is also used more loosely for neighboring methods that modify proposal kernels, mutation kernels, resampling laws, or intermediate targets to reduce mismatch between particle trajectories and the target distribution, but several of these methods are not control-theoretic cSMC in the strict sense (Gu et al., 2015, Peters et al., 2008, Lamberti et al., 2016).

1. Formal setting and scope

Controlled Sequential Monte Carlo is most naturally stated in the Feynman–Kac framework. Let $X_{0:T}=(X_0,\dots,X_T)$ be a non-homogeneous Markov chain on $(\mathsf X,\mathcal X)$ with law

$$\mathbb Q(\mathrm d x_{0:T}) = \mu(\mathrm d x_0)\prod_{t=1}^T M_t(x_{t-1},\mathrm d x_t),$$

and let strictly positive potentials $G_0\in\mathcal B(\mathsf X)$ and $G_t\in\mathcal B(\mathsf X\times\mathsf X)$, $t\ge 1$, define the path measure

$$\mathbb P(\mathrm d x_{0:T}) = Z^{-1} G_0(x_0)\prod_{t=1}^T G_t(x_{t-1},x_t)\,\mathbb Q(\mathrm d x_{0:T}),$$

with normalizing constant

$$Z=\mathbb E_{\mathbb Q}\!\left[G_0(X_0)\prod_{t=1}^T G_t(X_{t-1},X_t)\right].$$

The associated unnormalized and normalized marginals are

$$\gamma_t(\varphi)=\mathbb E_{\mathbb Q}\!\left[\varphi(X_t)G_0(X_0)\prod_{s=1}^t G_s(X_{s-1},X_s)\right],\qquad \eta_t(\varphi)=\frac{\gamma_t(\varphi)}{Z_t},$$

with $Z_t=\gamma_t(\mathsf X)$ and final target $(\mathsf X,\mathcal X)$0 (Heng et al., 2017).

This formulation covers both state-space and static models. In state-space models, $(\mathsf X,\mathcal X)$1 is the smoothing distribution and $(\mathsf X,\mathcal X)$2 the marginal likelihood. In static models, $(\mathsf X,\mathcal X)$3 is the target posterior or target distribution, obtained via a sequence of bridging measures (Heng et al., 2017). Tutorial treatments place cSMC within the broader auxiliary or twisted SMC methods literature and emphasize that proposal distributions and intermediate target distributions are the two main user design choices of SMC (Naesseth et al., 2019). The general SMC-sampler substrate is the sequence $(\mathsf X,\mathcal X)$4, forward kernels $(\mathsf X,\mathcal X)$5, backward kernels $(\mathsf X,\mathcal X)$6, and incremental weight

$(\mathsf X,\mathcal X)$7

which is the basic mechanism from which more elaborate twisted and controlled constructions are built (Dai et al., 2020).

2. Twisting, optimal control, and the zero-variance benchmark

The distinctive move in cSMC is to introduce a policy $(\mathsf X,\mathcal X)$8 of positive functions and to twist the proposal path measure. Given any path measure

$(\mathsf X,\mathcal X)$9

its $$\mathbb Q(\mathrm d x_{0:T}) = \mu(\mathrm d x_0)\prod_{t=1}^T M_t(x_{t-1},\mathrm d x_t),$$0-twisted version is

$$\mathbb Q(\mathrm d x_{0:T}) = \mu(\mathrm d x_0)\prod_{t=1}^T M_t(x_{t-1},\mathrm d x_t),$$1

where

$$\mathbb Q(\mathrm d x_{0:T}) = \mu(\mathrm d x_0)\prod_{t=1}^T M_t(x_{t-1},\mathrm d x_t),$$2

Applied to $$\mathbb Q(\mathrm d x_{0:T}) = \mu(\mathrm d x_0)\prod_{t=1}^T M_t(x_{t-1},\mathrm d x_t),$$3, this yields a controlled proposal path measure $$\mathbb Q(\mathrm d x_{0:T}) = \mu(\mathrm d x_0)\prod_{t=1}^T M_t(x_{t-1},\mathrm d x_t),$$4 (Heng et al., 2017).

Under $$\mathbb Q(\mathrm d x_{0:T}) = \mu(\mathrm d x_0)\prod_{t=1}^T M_t(x_{t-1},\mathrm d x_t),$$5, the same target $$\mathbb Q(\mathrm d x_{0:T}) = \mu(\mathrm d x_0)\prod_{t=1}^T M_t(x_{t-1},\mathrm d x_t),$$6 can be rewritten with twisted potentials: $$\mathbb Q(\mathrm d x_{0:T}) = \mu(\mathrm d x_0)\prod_{t=1}^T M_t(x_{t-1},\mathrm d x_t),$$7

$$\mathbb Q(\mathrm d x_{0:T}) = \mu(\mathrm d x_0)\prod_{t=1}^T M_t(x_{t-1},\mathrm d x_t),$$8

$$\mathbb Q(\mathrm d x_{0:T}) = \mu(\mathrm d x_0)\prod_{t=1}^T M_t(x_{t-1},\mathrm d x_t),$$9

The terminal normalizing constant is preserved: $G_0\in\mathcal B(\mathsf X)$0 Hence cSMC changes the simulation law without changing the target (Heng et al., 2017).

The optimal policy is characterized by a backward recursion. If $G_0\in\mathcal B(\mathsf X)$1 is a current policy and $G_0\in\mathcal B(\mathsf X)$2 is an additional twist, the optimal $G_0\in\mathcal B(\mathsf X)$3 solves

$G_0\in\mathcal B(\mathsf X)$4

with recursion

$G_0\in\mathcal B(\mathsf X)$5

$G_0\in\mathcal B(\mathsf X)$6

$G_0\in\mathcal B(\mathsf X)$7

Equivalently, the optimal value functions $G_0\in\mathcal B(\mathsf X)$8 satisfy a Bellman recursion (Heng et al., 2017).

This recursion yields the zero-variance benchmark. If $G_0\in\mathcal B(\mathsf X)$9, then $G_t\in\mathcal B(\mathsf X\times\mathsf X)$0, the twisted marginals coincide with the true marginals of $G_t\in\mathcal B(\mathsf X\times\mathsf X)$1, and

$G_t\in\mathcal B(\mathsf X\times\mathsf X)$2

Substituting $G_t\in\mathcal B(\mathsf X\times\mathsf X)$3 into the twisted potentials gives

$G_t\in\mathcal B(\mathsf X\times\mathsf X)$4

so all weights are constant (Heng et al., 2017). In state-space language, this is the controlled analogue of a fully adapted particle system; in general Feynman–Kac language, it is the ideal twisted path measure.

3. Approximate dynamic programming, policy classes, and online extensions

Exact optimal policies are generally intractable, so practical cSMC replaces the Bellman recursion by approximate dynamic programming. The basic algorithm starts from

$G_t\in\mathcal B(\mathsf X\times\mathsf X)$5

which corresponds to uncontrolled SMC, runs a $G_t\in\mathcal B(\mathsf X\times\mathsf X)$6-twisted SMC pass, and then regresses the value functions $G_t\in\mathcal B(\mathsf X\times\mathsf X)$7 on a tractable function class $G_t\in\mathcal B(\mathsf X\times\mathsf X)$8 (Heng et al., 2017). At time $G_t\in\mathcal B(\mathsf X\times\mathsf X)$9, one solves

$t\ge 1$0

and backward in time one fits

$t\ge 1$1

where

$t\ge 1$2

The policy is refined multiplicatively,

$t\ge 1$3

and the residuals determine how close the new twisted potentials are to constants (Heng et al., 2017).

This ADP view is central: cSMC is approximate dynamic programming for a finite-horizon KL control problem on path space (Heng et al., 2017). Function classes include quadratic functions in state-space models, quadratic-plus-likelihood terms for static models, radial basis functions for multimodal policies, and linear least-squares basis expansions (Heng et al., 2017). The analysis introduces Bellman semigroups, residual-based KL discrepancy bounds, semigroup stability constants, and a CLT for the estimated policy under linear least-squares classes (Heng et al., 2017).

Application-specific implementations exploit tractable policy classes. In diffusion-process inference via splitting schemes, the controlled proposal is built on conditionally Gaussian kernels and quadratic log-policies

$t\ge 1$4

with

$t\ge 1$5

Because multiplying a Gaussian kernel by $t\ge 1$6 yields another Gaussian kernel, $t\ge 1$7 is tractable and $t\ge 1$8 can be computed analytically (Huang et al., 19 Jul 2025). In that setting, cSMC is applied to Feynman–Kac representations of pseudolikelihoods arising from semi-linear SDEs, partial observation, and bridge augmentation.

Recent online work extends the offline smoothing-oriented formulation to real-time hidden Markov models through Online Rolling Controlled Sequential Monte Carlo (ORCSMC) (Xue et al., 1 Aug 2025). ORCSMC defines a rolling window of length $t\ge 1$9,

$$\mathbb P(\mathrm d x_{0:T}) = Z^{-1} G_0(x_0)\prod_{t=1}^T G_t(x_{t-1},x_t)\,\mathbb Q(\mathrm d x_{0:T}),$$0

and uses two particle systems: a learning filter that estimates twisting functions over the current window and an estimation filter that delivers filtering, smoothing, and marginal-likelihood output (Xue et al., 1 Aug 2025). The twisted model is expressed through positive functions $$\mathbb P(\mathrm d x_{0:T}) = Z^{-1} G_0(x_0)\prod_{t=1}^T G_t(x_{t-1},x_t)\,\mathbb Q(\mathrm d x_{0:T}),$$1,

$$\mathbb P(\mathrm d x_{0:T}) = Z^{-1} G_0(x_0)\prod_{t=1}^T G_t(x_{t-1},x_t)\,\mathbb Q(\mathrm d x_{0:T}),$$2

with corresponding twisted potentials

$$\mathbb P(\mathrm d x_{0:T}) = Z^{-1} G_0(x_0)\prod_{t=1}^T G_t(x_{t-1},x_t)\,\mathbb Q(\mathrm d x_{0:T}),$$3

and with local backward recursion targets

$$\mathbb P(\mathrm d x_{0:T}) = Z^{-1} G_0(x_0)\prod_{t=1}^T G_t(x_{t-1},x_t)\,\mathbb Q(\mathrm d x_{0:T}),$$4

For fixed lag $$\mathbb P(\mathrm d x_{0:T}) = Z^{-1} G_0(x_0)\prod_{t=1}^T G_t(x_{t-1},x_t)\,\mathbb Q(\mathrm d x_{0:T}),$$5, particle count $$\mathbb P(\mathrm d x_{0:T}) = Z^{-1} G_0(x_0)\prod_{t=1}^T G_t(x_{t-1},x_t)\,\mathbb Q(\mathrm d x_{0:T}),$$6, and iteration count $$\mathbb P(\mathrm d x_{0:T}) = Z^{-1} G_0(x_0)\prod_{t=1}^T G_t(x_{t-1},x_t)\,\mathbb Q(\mathrm d x_{0:T}),$$7, the method enforces bounded per-observation computation and memory (Xue et al., 1 Aug 2025).

4. Neighboring methodologies and historical lineage

The strict control-theoretic meaning of cSMC should be distinguished from several adjacent literatures that pursue related goals. Neural Adaptive Sequential Monte Carlo (NASMC) learns a forward proposal family by minimizing the inclusive KL divergence

$$\mathbb P(\mathrm d x_{0:T}) = Z^{-1} G_0(x_0)\prod_{t=1}^T G_t(x_{t-1},x_t)\,\mathbb Q(\mathrm d x_{0:T}),$$8

with particle-weighted gradients of the form

$$\mathbb P(\mathrm d x_{0:T}) = Z^{-1} G_0(x_0)\prod_{t=1}^T G_t(x_{t-1},x_t)\,\mathbb Q(\mathrm d x_{0:T}),$$9

It is aligned in goal with controlled SMC because it modifies particle propagation so trajectories better match the posterior and importance weights become less variable, but it is not standard controlled SMC: there is no Bellman equation, no explicit control cost plus terminal reward, no twisted path measure via optimal potentials, and no backward information recursion (Gu et al., 2015).

An earlier precursor is the SMC sampler with partial rejection control, which modifies the mutation step through a threshold $$Z=\mathbb E_{\mathbb Q}\!\left[G_0(X_0)\prod_{t=1}^T G_t(X_{t-1},X_t)\right].$$0 and induces a new effective forward kernel

$$Z=\mathbb E_{\mathbb Q}\!\left[G_0(X_0)\prod_{t=1}^T G_t(X_{t-1},X_t)\right].$$1

The paper proves the stagewise variance inequality

$$Z=\mathbb E_{\mathbb Q}\!\left[G_0(X_0)\prod_{t=1}^T G_t(X_{t-1},X_t)\right].$$2

so it is a clear example of proposal-kernel modification aimed at reducing weight variance, although it does not formulate a global optimal control problem (Peters et al., 2008).

Another neighboring direction controls the resampling law rather than the proposal law. Independent resampling Sequential Monte Carlo replaces the usual dependent multinomial/bootstrap resampling by a construction that preserves the same one-particle marginal law as classical resampling, while removing the induced dependence between resampled particles (Lamberti et al., 2016). In the static setting, the paper proves

$$Z=\mathbb E_{\mathbb Q}\!\left[G_0(X_0)\prod_{t=1}^T G_t(X_{t-1},X_t)\right].$$3

and

$$Z=\mathbb E_{\mathbb Q}\!\left[G_0(X_0)\prod_{t=1}^T G_t(X_{t-1},X_t)\right].$$4

so removing coupling lowers variance while preserving the target-oriented marginal effect. This enlarges the broader design space of “controlled SMC” by showing that one can control the joint resampling law itself (Lamberti et al., 2016).

Rare-event SISR methods offer an even earlier control perspective. In large-deviation problems, the resampling weights are chosen so that

$$Z=\mathbb E_{\mathbb Q}\!\left[G_0(X_0)\prod_{t=1}^T G_t(X_{t-1},X_t)\right].$$5

and in the simplest exponential-tilting case

$$Z=\mathbb E_{\mathbb Q}\!\left[G_0(X_0)\prod_{t=1}^T G_t(X_{t-1},X_t)\right].$$6

The resulting estimators are shown to be logarithmically efficient, with criteria such as

$$Z=\mathbb E_{\mathbb Q}\!\left[G_0(X_0)\prod_{t=1}^T G_t(X_{t-1},X_t)\right].$$7

so the control acts on particle allocation through resampling rather than on the proposal kernel itself (Chan et al., 2012).

Sequentially Constrained Monte Carlo (SCMC) is closer to tempered SMC samplers than to cSMC proper. It defines a sequence of targets by progressively enforcing a difficult constraint, for example

$$Z=\mathbb E_{\mathbb Q}\!\left[G_0(X_0)\prod_{t=1}^T G_t(X_{t-1},X_t)\right].$$8

or, for monotone regression,

$$Z=\mathbb E_{\mathbb Q}\!\left[G_0(X_0)\prod_{t=1}^T G_t(X_{t-1},X_t)\right].$$9

SCMC therefore contributes a path-design principle for constrained targets, but not a learned twisting or dynamic-programming control law (Golchi et al., 2014).

5. Applications and empirical record

The original cSMC paper reports substantial gains over state-of-the-art methods at a fixed computational complexity on a variety of applications, including state-space models and complex static models (Heng et al., 2017). Subsequent work specialized the framework to diffusion pseudolikelihoods under partial observation, bridge augmentation, and hypoelliptic dynamics (Huang et al., 19 Jul 2025), and to online filtering through rolling-window control adaptation (Xue et al., 1 Aug 2025).

Setting	Baseline / configuration	Reported result
Neuroscience state-space model	PMMH ESS, BPF vs cSMC	BPF: $$\gamma_t(\varphi)=\mathbb E_{\mathbb Q}\!\left[\varphi(X_t)G_0(X_0)\prod_{s=1}^t G_s(X_{s-1},X_s)\right],\qquad \eta_t(\varphi)=\frac{\gamma_t(\varphi)}{Z_t},$$0; cSMC: $$\gamma_t(\varphi)=\mathbb E_{\mathbb Q}\!\left[\varphi(X_t)G_0(X_0)\prod_{s=1}^t G_s(X_{s-1},X_s)\right],\qquad \eta_t(\varphi)=\frac{\gamma_t(\varphi)}{Z_t},$$1 (Heng et al., 2017)
Log-Gaussian Cox process ($$\gamma_t(\varphi)=\mathbb E_{\mathbb Q}\!\left[\varphi(X_t)G_0(X_0)\prod_{s=1}^t G_s(X_{s-1},X_s)\right],\qquad \eta_t(\varphi)=\frac{\gamma_t(\varphi)}{Z_t},$$2)	standard AIS, adaptive AIS	Variance of log-marginal-likelihood estimates $$\gamma_t(\varphi)=\mathbb E_{\mathbb Q}\!\left[\varphi(X_t)G_0(X_0)\prod_{s=1}^t G_s(X_{s-1},X_s)\right],\qquad \eta_t(\varphi)=\frac{\gamma_t(\varphi)}{Z_t},$$3 smaller than standard AIS, $$\gamma_t(\varphi)=\mathbb E_{\mathbb Q}\!\left[\varphi(X_t)G_0(X_0)\prod_{s=1}^t G_s(X_{s-1},X_s)\right],\qquad \eta_t(\varphi)=\frac{\gamma_t(\varphi)}{Z_t},$$4 smaller than adaptive AIS; MSE of adaptive AIS $$\gamma_t(\varphi)=\mathbb E_{\mathbb Q}\!\left[\varphi(X_t)G_0(X_0)\prod_{s=1}^t G_s(X_{s-1},X_s)\right],\qquad \eta_t(\varphi)=\frac{\gamma_t(\varphi)}{Z_t},$$5 larger than cSMC (Heng et al., 2017)
Partially observed hypoelliptic FHN	PMMH with cSMC (10 particles) vs BPF (125 particles)	cSMC was about twice as fast and had negligible likelihood-estimate dispersion compared with the BPF (Huang et al., 19 Jul 2025)
Online HMM inference	standard particle filtering approaches	Improved estimation accuracy and robustness in higher dimensions (Xue et al., 1 Aug 2025)

The state-space results in the original cSMC paper are not limited to PMMH. In a low-noise, partially observed Lorenz-96 system, cSMC sharply reduces relative variance of log marginal-likelihood estimates across parameter settings, with strongest improvements in difficult regimes such as low observation noise, misspecified parameters, and higher dimensions (Heng et al., 2017). In Bayesian logistic regression on three real datasets, cSMC achieves ESS near $$\gamma_t(\varphi)=\mathbb E_{\mathbb Q}\!\left[\varphi(X_t)G_0(X_0)\prod_{s=1}^t G_s(X_{s-1},X_s)\right],\qquad \eta_t(\varphi)=\frac{\gamma_t(\varphi)}{Z_t},$$6 and variance/RMSE reductions in $$\gamma_t(\varphi)=\mathbb E_{\mathbb Q}\!\left[\varphi(X_t)G_0(X_0)\prod_{s=1}^t G_s(X_{s-1},X_s)\right],\qquad \eta_t(\varphi)=\frac{\gamma_t(\varphi)}{Z_t},$$7 estimation, often by several orders of magnitude (Heng et al., 2017). In a classic nonlinear multimodal filtering benchmark, cSMC with RBF policy classes strongly improves ESS and reduces variance of $$\gamma_t(\varphi)=\mathbb E_{\mathbb Q}\!\left[\varphi(X_t)G_0(X_0)\prod_{s=1}^t G_s(X_{s-1},X_s)\right],\qquad \eta_t(\varphi)=\frac{\gamma_t(\varphi)}{Z_t},$$8, especially at high signal-to-noise ratio (Heng et al., 2017).

The diffusion-inference literature shows that cSMC is particularly valuable when the likelihood itself is only available through a high-dimensional Feynman–Kac representation. In semi-linear SDEs discretized by splitting schemes, cSMC is used because naive bootstrap particle filtering can have enormous variance, especially when observations are informative or bridge dimensions are large (Huang et al., 19 Jul 2025). Bridge augmentation reduces discretization bias and, under Assumption 2, the bridged transition converges in $$\gamma_t(\varphi)=\mathbb E_{\mathbb Q}\!\left[\varphi(X_t)G_0(X_0)\prod_{s=1}^t G_s(X_{s-1},X_s)\right],\qquad \eta_t(\varphi)=\frac{\gamma_t(\varphi)}{Z_t},$$9 to the true transition density as the number of bridge steps grows (Huang et al., 19 Jul 2025). In practice, cSMC converged in about $Z_t=\gamma_t(\mathsf X)$0–$Z_t=\gamma_t(\mathsf X)$1 iterations on bridged examples, around $Z_t=\gamma_t(\mathsf X)$2 particles per cSMC iteration were sufficient for stable estimates in SPSA measurements, and PMMH on the FHN example used $Z_t=\gamma_t(\mathsf X)$3 particles for cSMC versus $Z_t=\gamma_t(\mathsf X)$4 for BPF (Huang et al., 19 Jul 2025).

6. Limitations, misconceptions, and current directions

The formal analysis of cSMC is strong but not assumption-free. The theory relies on bounded potentials, suitable measurability and integrability, invertible Gram matrices for linear regression, smooth dependence of $Z_t=\gamma_t(\mathsf X)$5 on $Z_t=\gamma_t(\mathsf X)$6, CLTs for particle empirical measures, and contraction or Lipschitz conditions for iterated ADP (Heng et al., 2017). Practical failure modes include poor function classes, intractable twisted kernels, ill-conditioned regressions, approximation error accumulation in a single backward pass, and contraction assumptions that may fail globally (Heng et al., 2017).

Domain-specific applications introduce additional caveats. In diffusion inference, bridge augmentation still increases cost substantially; if the numerical scheme itself is unstable, cSMC can break because policy learning is corrupted; and Strang requires invertibility of $Z_t=\gamma_t(\mathsf X)$7, sometimes only available after refining the time step via bridging (Huang et al., 19 Jul 2025). ORCSMC is explicitly a finite-horizon online approximation to offline cSMC: it preserves bounded per-observation computation and memory by restricting adaptation to a rolling window, but therefore sacrifices global offline optimality (Xue et al., 1 Aug 2025).

A recurrent misconception is to identify cSMC with any adaptive SMC scheme. NASMC is proposal adaptation for SMC, not control-theoretic SMC; its alignment with cSMC is partial, with strong overlap in goal but not in formalism (Gu et al., 2015). Independent resampling modifies the joint law of the resampling step while keeping the same single-particle marginal, so it is best viewed as control of offspring coupling rather than control of the proposal path law (Lamberti et al., 2016). SCMC progressively imposes a difficult constraint by redesigning the intermediate targets, which makes it closely related to tempered SMC samplers rather than to optimal-control or twisting-based cSMC (Golchi et al., 2014).

The contemporary literature therefore supports a layered interpretation. In the strict sense, controlled sequential Monte Carlo is the twisted Feynman–Kac, KL-control, Bellman-recursion framework introduced for general state-space and static models (Heng et al., 2017). In a broader historical and methodological sense, the field also includes proposal adaptation, mutation-kernel control, resampling control, and sequential target design, all directed at the same central pathology: variance and degeneracy caused by mismatch between particle dynamics and the target distribution (Gu et al., 2015, Peters et al., 2008, Lamberti et al., 2016, Golchi et al., 2014).