Recurrent Explicit Duration SLDS

• REDSLDS is a time series model that extends SLDS by incorporating explicit duration variables to enable non-geometric and learnable dwell-time laws.
• It employs recurrent, state-dependent logistic-stick-breaking functions that couple the continuous latent states with switching and duration dynamics for improved regime segmentation.
• Using a Bayesian framework with Pólya–Gamma augmentation, REDSLDS achieves scalable inference and superior segmentation performance across diverse benchmarks.

The Recurrent Explicit Duration Switching Linear Dynamical System (REDSLDS) is a class of time series models that extends the classical Switching Linear Dynamical System (SLDS) by incorporating explicit duration variables and recurrent dependencies between discrete switches and the underlying continuous latent state. This design enables precise modeling of regime dynamics with non-geometric dwell-time distributions, and it allows switching and duration probabilities to depend in a recurrent manner on the continuous state trajectory. The Bayesian variant of REDSLDS leverages Pólya–Gamma augmentation for conjugate Gibbs sampling in the presence of non-linear, recurrently parameterized switching and duration distributions, enabling scalable sampling-based inference and parameter learning across a wide range of complex dynamical systems (Słupiński et al., 2024).

1. Model Specification and Generative Structure

REDSLDS augments the SLDS framework through two main mechanisms:

• Explicit duration modeling: A discrete countdown variable $d_t\in\{1,\dots,D_{\max}\}$ encodes the remaining number of time steps before a regime change, making durations non-geometric and allowing for arbitrarily specified (and learnable) dwell-time laws per regime.
• Recurrent, state-dependent switching and duration: Switching probabilities and duration distributions explicitly depend on the previous continuous latent state $x_{t-1}$ via logistic-stick-breaking link functions parameterized by weights $\{R^S_k,r^S_k\}_{k=1}^K$ and $\{R^D_k,r^D_k\}_{k=1}^K$.

The random variables are:

• $s_t\in\{1,\dots,K\}$: discrete regime at time $t$
• $d_t\in\{1,\dots,D_{\max}\}$: countdown duration in regime $s_t$
• $x_t\in\mathbb R^M$: continuous latent state
• $y_t\in\mathbb R^N$: observed emission

The complete joint over a sequence $x_{t-1}$0 is

$x_{t-1}$1

where each factor is fully parameterized to allow recurrence and explicit duration (Słupiński et al., 2024).

For $x_{t-1}$2, $x_{t-1}$3 deterministically and $x_{t-1}$4. When $x_{t-1}$5, both the new regime $x_{t-1}$6 and the new duration $x_{t-1}$7 are drawn jointly, each with stick-breaking logistic weights depending on $x_{t-1}$8. This produces regime durations with arbitrary, non-geometric distributions and enables direct recurrent control over regime switching.

2. Parameterization and Priors

The model parameters $x_{t-1}$9 include:

• Linear-dynamical evolution: For each regime $\{R^S_k,r^S_k\}_{k=1}^K$0, matrices $\{R^S_k,r^S_k\}_{k=1}^K$1 for $\{R^S_k,r^S_k\}_{k=1}^K$2
• Emission distributions: For each regime $\{R^S_k,r^S_k\}_{k=1}^K$3, $\{R^S_k,r^S_k\}_{k=1}^K$4 for $\{R^S_k,r^S_k\}_{k=1}^K$5
• Initial distributions: Categorical $\{R^S_k,r^S_k\}_{k=1}^K$6, regime-specific duration priors $\{R^S_k,r^S_k\}_{k=1}^K$7, and Gaussian $\{R^S_k,r^S_k\}_{k=1}^K$8
• Recurrent logit weights: $\{R^S_k,r^S_k\}_{k=1}^K$9 and $\{R^D_k,r^D_k\}_{k=1}^K$0 for stick-breaking logistic links controlling regime and duration transitions

Priors for the linear dynamical parameters are Matrix-Normal-Inverse-Wishart, and the recurrent weights $\{R^D_k,r^D_k\}_{k=1}^K$1, $\{R^D_k,r^D_k\}_{k=1}^K$2 are given multivariate Gaussians (Słupiński et al., 2024).

3. Bayesian Inference: Pólya–Gamma Augmentation and Blocked Gibbs

Inference in REDSLDS is complicated by the non-conjugate nature of the stick-breaking parameterizations for both regime and duration transitions. Direct Gibbs sampling is infeasible for these components due to logistic–Gaussian non-conjugacy.

To address this, REDSLDS employs Pólya–Gamma augmentation [Polson, Scott & Windle, 2013], which replaces logistic link likelihoods with auxiliary variables $\{R^D_k,r^D_k\}_{k=1}^K$3 such that, conditional on $\{R^D_k,r^D_k\}_{k=1}^K$4, the likelihood is quadratic in the stick-breaking logits. This enables block-Gibbs updates for both the recurrent weights and the latent states, preserving tractability:

1. Sample discrete paths $\{R^D_k,r^D_k\}_{k=1}^K$5 jointly by Forward–Filtering Backward–Sampling (FFBS), conditional on latent variables and augmented variables.
2. Sample Pólya–Gamma variables for each stick-breaking link (one set for regime, one for duration), producing conditional Gaussian likelihoods for the logit weights.
3. Sample continuous latent state $\{R^D_k,r^D_k\}_{k=1}^K$6 using specialized Kalman smoothing with added quadratic forms from the Pólya–Gamma variables.
4. Sample model parameters from posterior conjugate distributions.

All updates can be performed as direct samples from standard distributions or multivariate Gaussians, bypassing the need for Metropolis adjustment or rejection steps. This yields efficient, rapidly mixing MCMC chains (Słupiński et al., 2024).

4. Training and Parameter Estimation

REDSLDS uses a fully Bayesian MCMC training scheme. Each Gibbs cycle consists of:

• Sampling latent trajectories (states, regimes, durations, auxiliary Pólya–Gamma variables)
• Sampling all model parameters from their conjugate posterior conditionals

For the linear dynamical parameters in regime $\{R^D_k,r^D_k\}_{k=1}^K$7, data points $\{R^D_k,r^D_k\}_{k=1}^K$8 are collected, and the posterior is computed using sufficient statistics, yielding a Matrix-Normal-Inverse-Wishart posterior. For switching and duration logit weights, the Gaussian prior and likelihood terms (from Pólya–Gamma augmentation) yield closed-form Gaussians for block updates.

This approach obviates the need for Expectation-Maximization and instead yields samples from the full joint posterior over latent paths and model parameters (Słupiński et al., 2024).

5. Computational Aspects and Algorithmic Complexity

The explicit duration structure requires storing variables of size $\{R^D_k,r^D_k\}_{k=1}^K$9 at each time step. FFBS for the discrete variables operates in $s_t\in\{1,\dots,K\}$0 per sequence, and the continuous state smoothing leverages the information filter with quadratic (pseudo-Gaussian) potentials from the Pólya–Gamma variables.

The blocked Gibbs scheme ensures all steps remain tractable and avoids bottlenecks associated with non-conjugate updates found in alternatives such as rSLDS. All matrix computations (for priors and sufficient statistics) exploit standard numerical linear algebra, and the augmentation introduces only moderate overhead.

6. Empirical Results and Benchmark Evaluation

The performance of REDSLDS has been demonstrated on both quantitative and qualitative benchmarks (Słupiński et al., 2024):

• NASCAR (simulated 4-regime): On train-test splits of sequences, REDSLDS achieves weighted F1 scores of $s_t\in\{1,\dots,K\}$1, a significant improvement (10–30 points) over rSLDS, and shows higher held-out likelihood especially for short segments.
• Honeybee waggle dance: On six dance recordings, REDSLDS attains framewise accuracy $s_t\in\{1,\dots,K\}$2 (vs. rSLDS $s_t\in\{1,\dots,K\}$3), weighted F1 $s_t\in\{1,\dots,K\}$4, and improved test log-likelihood. REDSLDS avoids severe over-segmentation and accurately recovers the set of dance primitives.
• Mouse Behavior (BehaveNet): The rSLDS collapses almost all sequences into a single state, while REDSLDS correctly identifies multiple behavioral regimes with distinct amplitude patterns and plausible dwell times.

Summary statistics appear in the tables below.

Dataset	rSLDS F1 / Accuracy	REDSLDS F1 / Accuracy
NASCAR, F1 (split=5)	$s_t\in\{1,\dots,K\}$5	$s_t\in\{1,\dots,K\}$6
Honeybee, Accuracy	$s_t\in\{1,\dots,K\}$7	$s_t\in\{1,\dots,K\}$8
Honeybee, W-F1	$s_t\in\{1,\dots,K\}$9	$t$0

Smoother and more persistent regime structure is observed in REDSLDS. Segment-stability and behavioral interpretability are also qualitatively improved compared to rSLDS.

7. Relevance, Impact, and Extensions

REDSLDS establishes a general-purpose Bayesian framework for switching time series with explicit, non-geometric durations and recurrent coupling between the discrete and continuous components. Key technical advances include:

• Tractable, exact block Gibbs for non-conjugate stick-breaking links via Pólya–Gamma augmentation
• Generalization of duration modeling beyond geometric laws, yielding better real-world segmentation and dwell-time recovery
• Bayesian estimation for all model parameters, supporting uncertainty quantification in regime segmentation

Empirical results demonstrate that explicit durations and recurrence provide substantial gains in segmentation fidelity and interpretability over standard rSLDS and SLDS models. These properties are especially pronounced in domains characterized by heterogeneous regime durations, such as behavioral data, robotics, and animal motion (Słupiński et al., 2024).

Future extensions could explore higher-dimensional switching processes, hierarchical duration models, or alternative neural parameterizations for the recurrent structure, but such developments are not present in the cited works.

---

For further technical detail, implementation, and derivations, consult "Bayesian Inference in Recurrent Explicit Duration Switching Linear Dynamical Systems" (Słupiński et al., 2024). For related deep-learning counterparts and alternative inference strategies, see "Deep Explicit Duration Switching Models for Time Series" (Ansari et al., 2021).