Identification of jump Markov linear models using particle filters

Abstract: Jump Markov linear models consists of a finite number of linear state space models and a discrete variable encoding the jumps (or switches) between the different linear models. Identifying jump Markov linear models makes for a challenging problem lacking an analytical solution. We derive a new expectation maximization (EM) type algorithm that produce maximum likelihood estimates of the model parameters. Our development hinges upon recent progress in combining particle filters with Markov chain Monte Carlo methods in solving the nonlinear state smoothing problem inherent in the EM formulation. Key to our development is that we exploit a conditionally linear Gaussian substructure in the model, allowing for an efficient algorithm.

• The paper introduces a Rao-Blackwellized PSAEM algorithm that employs particle filtering and MCMC for effective identification of JMLMs.
• The algorithm leverages the conditionally linear Gaussian structure and Kalman filters to reduce computational complexity.
• Numerical experiments demonstrate faster convergence and higher accuracy compared to traditional methods in single- and multi-dimensional systems.

Identification of Jump Markov Linear Models Using Particle Filters

This essay provides a detailed examination of the techniques employed for the identification of Jump Markov Linear Models (JMLMs) using particle filters, as presented in the arXiv paper "Identification of jump Markov linear models using particle filters". The focus is on the implementation of an Expectation Maximization (EM) algorithm augmented by advanced sampling methods to achieve maximum likelihood estimates for the parameters of these models.

Introduction to Jump Markov Linear Models

JMLMs consist of multiple linear state-space models and a discrete state representing the jumps between them. The primary challenge is identifying these models without a closed-form solution for the unknown jump sequence. The paper proposes an EM-based algorithm that leverages Particle Filters (PFs) and Markov Chain Monte Carlo (MCMC) methods to address the nonlinear state smoothing problem inherent in the EM framework. This approach efficiently exploits the conditionally linear Gaussian substructure of the model.

The Expectation Maximization Framework

The EM algorithm iteratively maximizes the likelihood by alternating between an expectation step (E-step) and a maximization step (M-step). For JMLMs, the state $x_{1:T}$ is latent, while $y_{1:T}$ is observed. The key challenge is approximating the intractable $Q(\theta, \theta')$ function. Sequential Monte Carlo (SMC) methods, such as the Particle Smoothing EM (PSEM), provide a framework for approximating this function. However, the paper extends this framework using a stochastic approximation approach, resulting in the Particle Stochastic Approximation EM (PSAEM) method.

Figure 1: Numerical example 1. Mean (lines) and 0.5 standard deviation (fields) $\mathcal{H}_2$ error for 7 runs of the RB-PSAEM using $N=3$ particles (black), PSAEM using $N=20$ particles (blue), and PSEM using $N=100$ particles and $M=20$ backward trajectories (red).

Monte Carlo Methods for Smoothing

Monte Carlo methods, specifically MCMC, are employed to address the smoothing of JMLMs. The Rao-Blackwellized approach exploits conditional linear Gaussian models to streamline computation. By decoupling the estimation of $s_{1:T}$ from $z_{1:T}$, the algorithm uses Kalman filters for the Gaussian segments and MCMC for the discrete state transitions. This leads to computational efficiency and improved convergence properties.

Implementation of Algorithm

The paper's primary contribution is the development of a Rao-Blackwellized PSAEM algorithm. The computational complexity is linear concerning both time steps $T$ and particle count $N$. Key steps include:

1. Initialization of parameters and state sequences.
2. Running the particle filter with ancestor sampling to generate state trajectories.
3. Updating the sufficient statistics for parameter estimation.
4. Iterative maximization using stochastic approximation to refine model parameters.

Numerical Experiments

The paper validates the proposed algorithm through simulations on both single-dimensional and multi-dimensional systems. The results demonstrate superior performance compared to previous methods in terms of convergence speed and estimate accuracy.

Figure 2: Mean $\mathcal{H}_2$ error for each mode in a multi-dimensional system.

Conclusion

The development of an EM-based algorithm for JMLMs using particle filters presents a robust framework for parameter estimation in models characterized by discrete jumps. By leveraging Rao-Blackwellization and modern Monte Carlo techniques, the proposed method achieves efficient and accurate model identification. Future work may explore extending these methodologies to broader classes of hybrid systems and investigating their applicability in more diverse domains.