---
title: Particle Gibbs with Ancestor Sampling (PGAS)
url: https://www.emergentmind.com/topics/particle-gibbs-with-ancestor-sampling-pgas
type: topic
---

# Particle Gibbs with Ancestor Sampling (PGAS)

Particle Gibbs with Ancestor Sampling (PGAS) is an advanced Markov chain Monte Carlo (MCMC) algorithm designed for efficient joint inference of latent trajectories and static parameters in high-dimensional state-space models (SSMs). By combining conditional Sequential Monte Carlo (cSMC) with a backward-looking ancestor resampling step, PGAS greatly alleviates the path degeneracy and poor mixing characteristic of standard Particle Gibbs (PG) samplers in long sequences, non-Gaussian and nonlinear systems, and models with strong dependencies [1401.0604, 1304.1887, 1210.6911, 1910.14145]. The development, properties, and implementation of PGAS are summarized below.

## 1. Conditional SMC, PMCMC, and Context of PGAS

PGAS is a specific instance of the broader class of Particle MCMC (PMCMC) algorithms introduced by Andrieu, Doucet, and Holenstein, which embed Sequential Monte Carlo samplers as Gibbs moves within an MCMC framework to target the exact joint posterior over latent states and static parameters in complex statistical models. In PMCMC, sampling from the conditional distribution of the latent state sequence given parameters is replaced by running a cSMC sampler conditioned on a reference trajectory [1401.0604].

Conditional SMC (cSMC), the core ingredient of PG, proceeds by simulating an SMC particle filter in which one particular state sequence (the reference path) is held fixed. The resulting Markov kernel leaves the desired smoothing distribution invariant, but path degeneracy often impedes mixing as time horizon increases—the newly resampled trajectory frequently matches the old reference trajectory for a substantial prefix [1304.1887, 2007.15862].

## 2. Core PGAS Algorithm and Markov Kernel

PGAS augments cSMC with an ancestor sampling step at each time point, which randomly reassigns the ancestor of the retained trajectory according to a backward-weighted distribution. This procedure probabilistically breaks up the genealogy of the reference path and enables efficient exploration of latent trajectory space [1210.6911].

Let $x_{1:T}$ denote the latent state trajectory, $y_{1:T}$ the data, and $N$ the number of particles. Key steps of the PGAS kernel (in Markovian HMM or SSM context) are:

1. **Initialization**: At $t=1$, sample $N-1$ particles from the initial distribution, set the last particle to the reference. Compute and normalize weights.
2. **Forward Pass** $(t=2,\dots,T)$:
   - For $i=1,\dots,N-1$: sample ancestor $a_t^i$ according to normalized weights, propagate state $x_t^i \sim p(x_t|x_{t-1}^{a_t^i})$.
   - *Ancestor Sampling for Reference:* For $i=1,\dots,N$, compute backward weights $\tilde w_{t-1|T}^i \propto w_{t-1}^i\, p(x_t^{*}|x_{t-1}^i)$, where $x_t^{*}$ is the reference at $t$. Sample $a_t^N$ from this distribution, set $x_t^N = x_t^{*}$.
   - Update and normalize weights.
3. **Path Extraction**: Sample a new trajectory index according to terminal weights; reconstruct the full trajectory by tracing ancestor indices [1401.0604, 1304.1887, 1703.00968].

For non-Markovian models, the backward weights generalize to
\[
\tilde w_{t-1|T}^i \propto w_{t-1}^i\, \frac{ p(x'_{t:T}, y_{t:T}| x_{1:t-1}^i, y_{1:t-1}) }{ p(x_{1:t-1}^i, y_{1:t-1}) },
\]
which can require truncation or approximation in practice [1210.6911].

## 3. Theoretical Properties and Mixing

Ancestor sampling fundamentally mitigates the path degeneracy inherent in standard PG. By re-choosing parental lineages of the conditioned trajectory, PGAS increases the chance of moving off the previous reference path, thereby producing a much more rapidly mixing Markov kernel over latent trajectories even with modest particle counts (often $N=20$–$100$ suffices) [1304.1887, 1401.0604].

Main theoretical properties:
- **Uniform ergodicity**: Under mild regularity (e.g., bounded potentials), PGAS is uniformly ergodic for fixed $N$, with the contraction coefficient improving as $N$ increases [1304.1887].
- **Asymptotic variance reduction**: PGAS achieves strictly lower lag-1 autocorrelation and therefore smaller asymptotic variance in the CLT than standard Particle Gibbs [1304.1887].
- **Exact invariance**: The Markov kernel leaves the joint smoothing distribution and, if embedded in a full Gibbs scheme, the joint posterior over states and parameters, invariant [1210.6911, 1806.09548].

## 4. Implementation, Computational Complexity, and Tuning

Each PGAS iteration requires one conditional SMC pass of $N$ particles for $T$ time-steps. For Markovian models, the overall per-iteration computational cost is $O(NT)$. For non-Markovian models, backward-weights require additional computation, but efficient truncation strategies maintain linear or near-linear complexity [1210.6911, 1910.14145].

Guidelines and variations:
- Proposal distribution $q(x_t|x_{t-1},y_t)$ may be the prior or an optimal auxiliary form to control the importance weight variance. For nonlinear or non-Gaussian systems, Taylor or problem-specific proposals can further stabilize sampling [1703.00968].
- The number of particles $N$ is chosen to achieve satisfactory effective sample size (ESS) at each step. PGAS generally allows much smaller $N$ than PG for comparable mixing and accuracy [2501.03395].
- Systematic or residual resampling can be employed within cSMC to further reduce variance when ancestor sampling is infeasible [1304.1887].

## 5. Applications and Extensions

PGAS has been adopted in a diversity of domains, including:
- Bayesian inference in nonlinear and/or non-Gaussian state-space models (e.g. GEV models with Gaussian copula dependence) [1703.00968].
- Ecological population dynamics and epidemic models, where parameter elimination can be performed within PGAS when conjugacy permits, yielding marginalized PGAS (mPGAS) with substantially improved mixing and efficiency [1910.14145].
- Inference for infinite-state or nonparametric models, as in infinite Hidden Markov Models (iHMMs), exploiting optimized proposals within the PGAS kernel [1505.00428].
- Efficient smoothing of Markov jump processes and continuous-time Bayesian networks, even for infinite or very large discrete state spaces [1505.01434].
- Multiscale and multivariate hierarchical models, supporting joint learning of latent trajectories and hyperparameters [2408.06425].
- Probabilistic programming systems, where delayed-sampling and automatic conjugacy recognition enable automated deployment of PGAS and mPGAS [1910.14145, 1501.06769].

## 6. Empirical Performance and Diagnostics

Empirical studies consistently demonstrate that PGAS substantially outperforms standard PG in terms of trajectory mixing, autocorrelation, and robustness to particle number, especially as the time horizon $T$ becomes large. For instance, in Bayesian GEV copula models, PGAS with $N=1000$ maintained an "update frequency" (the proportion of MCMC sweeps where the reference path is altered) close to 99.9%—compared to rapid degeneracy and near-fixed paths in PG [1703.00968].

Key diagnostics reported in the literature include:
- Posterior credible bands on latent states closely tracking ground truth.
- Marginal inefficiency factors for global parameters that remain within the practicable range even in high dimensions and strongly dependent models.
- Consistently lower lag-1 autocorrelation and higher ESS per unit time for PGAS relative to PG and to alternative PMCMC variants [1304.1887, 1910.14145, 2501.03395].

PGAS can be combined with stochastic approximation EM, Rao-Blackwellized smoothing, and grid-based HMM approximations for further computational gains and scalability [1806.09548, 2501.03395, 1210.6911].

## 7. Model-Specific and Algorithmic Innovations

Recent extensions and innovations include grid-based PGAS for continuous-valued SSMs, marginalized PGAS exploiting conjugacy, and adaptive truncation strategies for non-Markovian ancestry weights. Detailed implementation recipes and pseudocode for all such variants are available in the cited works.

- Grid-PGAS leverages discrete HMM approximations to focus particle allocation effectively in high-posterior-density regions, yielding order-of-magnitude computational speedups in regime-switching and real-world forecasting models [2501.03395].
- Marginalized PGAS (mPGAS) uses exponential family and conjugacy structure to integrate out parameters during trajectory updates, minimizing autocorrelation and enabling automatable inference backends in probabilistic programming languages [1910.14145].
- Non-Markovian PGAS employs backward-weight truncation: under suitable decay of dependence, the theoretical impact of truncating ancestor weights decays exponentially and can be adaptively controlled [1210.6911].

PGAS therefore forms a generic, robust, and theoretically-grounded foundation for high-dimensional Bayesian filtering and smoothing, enabling scalable and precise inference across a wide variety of modern state-space models [1401.0604, 1703.00968, 1910.14145, 2501.03395].

Source: https://www.emergentmind.com/topics/particle-gibbs-with-ancestor-sampling-pgas