---
title: Gaussian Smoothing in CLG Models
url: https://www.emergentmind.com/topics/smoothing-in-conditionally-linear-gaussian-models
type: topic
---

# Gaussian Smoothing in CLG Models

A conditionally linear Gaussian (CLG) model describes stochastic dynamical systems where the state evolution and observation equations are linear and Gaussian in a subset of states, conditioned on the remaining (potentially nonlinear or discrete) states. Smoothing in this context refers to estimating the full trajectory of hidden states using all available observations over a fixed interval. Advanced CLG smoothing methods exploit conditional structure to combine analytical (Kalman-style) updates for linear subsystems and Monte Carlo or mixture techniques for the nonlinear/discrete components. This article provides a comprehensive treatment of Gaussian smoothing in CLG models, encompassing both the theoretical formulation and state-of-the-art smoothing algorithms.

## 1. Structure of Conditionally Linear Gaussian Models

A discrete-time CLG state-space model partitions the hidden state $x_k = [x_k^{(L)}; \, x_k^{(N)}]$ into a linear substate $x_k^{(L)}$ evolving via linear-Gaussian dynamics conditioned on the nonlinear or discrete substate $x_k^{(N)}$:

\[
\begin{aligned}
x_{k+1}^{(L)} &= A_k^{(L)}(x_k^{(N)})\,x_k^{(L)} + f_k^{(L)}(x_k^{(N)}) + w_k^{(L)} \\
x_{k+1}^{(N)} &= f_k^{(N)}(x_k^{(N)}) + A_k^{(N)}(x_k^{(N)})\,x_k^{(L)} + w_k^{(N)}
\end{aligned}
\]
\[
y_k = h_k(x_k^{(N)}) + B_k(x_k^{(N)})\,x_k^{(L)} + e_k
\]
where $w_k^{(L)}$ and $w_k^{(N)}$ are mutually independent zero-mean Gaussians. The nonlinear/“regime” component $x_k^{(N)}$ may be discrete or continuous; for example, in a Jump Markov Linear System (JMLS), the regime $r_k$ is discrete and $x_k$ is fully linear-Gaussian conditioned on $r_{1:T}$ [2004.08561, 1705.07598, 1505.06357, 1707.01311].

## 2. Smoothing Formulation in CLG Models

Given observations $y_{1:T}$, the fixed-interval smoothing objective is to compute $p(x_k|y_{1:T})$ or the joint $p(x_{1:T}|y_{1:T})$. The CLG structure enables marginalization of the linear subspace conditioned on a realization of the nonlinear/discrete trajectory, reducing variance and computational complexity—a technique known as Rao-Blackwellization [1505.06357, 1705.07598, 1707.01311].

The general solution exploits the factorization:
\[
p(x_k|y_{1:T}) \propto p(x_k|y_{1:k})\,p(y_{k+1:T}|x_k)
\]
where $p(x_k|y_{1:k})$ is available from forward filtering and $p(y_{k+1:T}|x_k)$ from backward information filtering. In fully linear Gaussian cases, these are propagated analytically; for general CLG models, Rao-Blackwellized particle smoothers and mixture-based approaches are employed.

## 3. Two-Filter and Hybrid Smoothing Algorithms

Several classes of algorithms target the CLG smoothing problem:

### 3.1. Exact Two-Filter Gaussian Mixture Smoother (JMLS, Discrete Regime CLG)

In Jump Markov Linear Systems, the smoother applies Kalman-style forward filtering and an information-form backward filter. The hybrid state $(x_k, r_k)$ admits a closed-form Gaussian mixture parameterization:
- Forward recursion: $p(x_k, r_k | y_{1:k})$ is a mixture over regimes.
- Backward information recursion: $p(y_{k+1:T} | x_k, r_k)$ is propagated as a Gaussian sum in information form.
- Smoother fusion: The product yields a mixture with component count growing as $M^T$ (regime multiplicity).

Without approximation, mixture cardinality becomes computationally intractable [2004.08561].

### 3.2. Rao-Blackwellized Particle Smoothing (General CLG)

For nonlinear regime chains or high-dimensional continuous $x_k^{(N)}$, Rao-Blackwellized particle smoothers (RBPS) are dominant:
- Represent trajectories of $x_{k}^{(N)}$ with a particle system.
- For each particle, propagate analytic Kalman filtering/smoothing for $x_k^{(L)}$.
- Backward smoothing is implemented via forward-filter-backward-simulation (FFBS) or two-filter decompositions, merging forward and backward messages to obtain smoothed estimates and sample trajectories [1505.06357, 1705.07598, 1707.01311].

### 3.3. Variants: Turbo Smoothing and Particle Rejuvenation

Turbo smoothing leverages parallel concatenation of forward and backward Bayesian filters on a factor graph, iteratively exchanging pseudo-measurement messages between extended Kalman filter (EKF) and particle filter (PF) modules in forward and backward passes. This framework can achieve tight complexity-accuracy trade-offs and significant memory savings [1902.05717]. Particle rejuvenation introduces additional backward sampling or resampling steps to mitigate path degeneracy in the discrete state, enlarging the support of the backward trajectories and reducing estimation variance [1707.01311].

## 4. Gaussian Mixture Explosion and Reduction

In mixture-based (e.g., JMLS) smoothers, the number of Gaussian mixture terms explodes exponentially with time. Specifically, $M^s_k = M^k M^{T-k}$ at time $k$, leading to $O(M^T)$ computational and storage complexity [2004.08561].

To make smoothing tractable:
- **Gaussian mixture reduction** merges or prunes components. The preferred method (KL-merge) merges the pair of Gaussians with minimal increase in KL divergence, using explicit moment-matching. This controls computational cost at the price of approximation error, tunable by the maximum mixture size $L$ [2004.08561].
- Trade-off: Larger $L$ decreases smoothing error and approaches the exact solution; smaller $L$ increases speed at the expense of approximation accuracy.

## 5. Computational Complexity and Accuracy Considerations

The complexity of exact smoothers is exponential in the length $T$ for mixture-based approaches on discrete regimes. With mixture reduction capped at $L$ components per mode, cost becomes $O(T L^2 + T L M)$ [2004.08561].

For Rao-Blackwellized particle smoothers handling continuous nonlinear states:
- Forward filtering: $O(T N_p d^3)$ for $N_p$ particles and linear subspace of dimension $d$.
- Backward smoothing: Comparable cost per sample trajectory.
- Total complexity: $O(N_p T)$ for marginal smoothing, or $O(M N_p T)$ for $M$ joint trajectories [1705.07598, 1505.06357].

Empirical results indicate that, for moderate mixture or particle budgets, smoothed estimates can achieve RMSE improvements of $20\text{--}40\%$ over filtering. Particle rejuvenation further reduces MSE for the discrete regime by up to $50\%$ in some settings [1707.01311]. Turbo smoothing demonstrates accuracy comparable to RBPS at reduced computation and memory [1902.05717].

## 6. Continuous-Time Linear Gaussian Smoothing

In continuous-time settings, the smoothing problem admits a pathwise characterization. The smoothing error evolves as an Ornstein-Uhlenbeck (OU) process driven by a backward Riccati equation for the error covariance, enabling exact Monte Carlo sampling of smoothed trajectories. Both Kalman-Bucy filtering and Rauch-Tung-Striebel (RTS) smoothing appear as marginals in this framework, and the Bryson-Frazier information-form smoother arises naturally as a corollary [2601.01805].

Key features include:
- The smoothed state trajectory is Gaussian with mean given by the backward-propagated filter and covariance determined by the OU smoothing error.
- Pathwise sampling of conditioned state processes is immediate once the Riccati equation is solved.

## 7. Practical Aspects and Algorithmic Summary

The following table summarizes key CLG smoothing approaches:

| Method                                  | Linearized States | Nonlinear/Discrete States | Complexity                |
|------------------------------------------|-------------------|--------------------------|---------------------------|
| Gaussian Mixture (JMLS) [2004.08561]     | Kalman            | Markov chain (regime)    | $O(M^T)$ (exact), $O(TL^2)$ with reduction |
| RB Particle Smoother [1505.06357, 1705.07598] | Kalman         | Particle representation  | $O(N_pT)$ (filter), $O(MN_pT)$ (M samples) |
| Turbo Smoother [1902.05717]              | EKF               | Particle (PF)            | $O(T[D^3+N_pD_N^2](1+N_{it}))$             |
| Particle Rejuvenation [1707.01311]       | Kalman            | Particle (with resampling) | $O(NJ m^3)$ per step with full support      |
| Pathwise OU (continuous-time) [2601.01805] | Kalman-Bucy      | — (fully linear-Gaussian)| Solve Riccati + pathwise sampling           |

Algorithm selection in practice is governed by trade-offs between model structure, required accuracy, memory and computational budget, and the size and nature of nonlinear/discrete subsystems.

## References

- "A New Smoothing Algorithm for Jump Markov Linear Systems" [2004.08561]
- "Rao-Blackwellized Particle Smoothing as Message Passing" [1705.07598]
- "Rao-Blackwellized particle smoothers for conditionally linear Gaussian models" [1505.06357]
- "Particle rejuvenation of Rao-Blackwellized Sequential Monte Carlo smoothers for Conditionally Linear and Gaussian models" [1707.01311]
- "A New Smoothing Technique based on the Parallel Concatenation of Forward/Backward Bayesian Filters: Turbo Smoothing" [1902.05717]
- "Pathwise Representation of the Smoothing Distribution in Continuous-Time Linear Gaussian Models" [2601.01805]

Source: https://www.emergentmind.com/topics/smoothing-in-conditionally-linear-gaussian-models