Switching Multiscale Dynamical System Models

• Switching multiscale dynamical system models are probabilistic frameworks that integrate regime-switching with multiscale latent dynamics to model non-stationary, multi-temporal processes across various fields.
• They employ discrete Markov chains, continuous latent state dynamics, and multimodal observation models to capture abrupt transitions and nested interactions at varied spatiotemporal scales.
• Learning and inference techniques such as EM, sequential Monte Carlo, and online causation entropy boosting enable robust regime detection and improved prediction accuracy.

Switching multiscale dynamical system (SMDS) models are probabilistic frameworks that integrate regime-switching mechanisms with multiscale latent structures to capture the complex, non-stationary, and multi-temporal behaviors of real-world systems. Such models accommodate abrupt transitions ('regime switches') between distinct dynamical laws, while simultaneously modeling interactions and structure across multiple spatiotemporal scales. SMDS models have been developed to address critical modeling challenges in neuroscience, geophysical dynamics, and other domains where both switching phenomena and multiscale processes are intrinsic.

1. Mathematical Formulations of SMDS

-Core constructs of SMDS frameworks typically include (i) a discrete regimen or 'switch' process, (ii) continuous multiscale latent dynamics (sometimes modality-specific), and (iii) heterogeneous observation models.

Discrete regime process:

A Markov chain governs the evolution of a latent discrete state $z_t \in \{1,\ldots,M\}$. At initialization, $z_1 \sim \pi$ (with $\pi \in \mathbb{R}^M$, $\sum_k \pi_k=1$). For $t \geq 2$, transitions $P(z_t = j | z_{t-1} = i) = A_{ij}$ (with $A \in \mathbb{R}^{M \times M}$, $\sum_j A_{ij} = 1$) impose regime-dependent structure (Kim et al., 14 Dec 2025, Vélez-Cruz et al., 24 Oct 2024).

Multiscale continuous latent states:

Each regime $k$ governs linear (or nonlinear) dynamics for the multiscale latent state. An example specification for $m$ modalities:

$$x_{t+1}^{(m)} = F_k^{(m)} x_t^{(m)} + q_t^{(m)}, \qquad q_t^{(m)} \sim \mathcal{N}(0, Q_k^{(m)})$$

Regimes may encode modality- or scale-specific latent state dimensions, with stacked or block-diagonal $F_k$, $Q_k$ for multiscale and multimodal fusion (Kim et al., 14 Dec 2025).

Observation models:

• Continuous modalities (e.g., LFP): $$y_t^{(c)} | x_t^{(c)}, z_t = k \sim \mathcal{N}(H_k^{(c)} x_t^{(c)}, R_k^{(c)})$$
• Discrete (e.g., spikes): $$y_t^{(s)} | x_t^{(s)}, z_t = k \sim \mathrm{Poisson}(\lambda_{t})$$, with $$\lambda_{t,c} = \exp[ H_k^{(s)} x_t^{(s)} + d_k^{(s)} ]$$.

Joint factorization:

For $T$ time steps, the joint probability over latent regimes, latent states, and observations is:

$$p(z_{1:T}, x_{1:T}, y_{1:T}) = \pi_{z_1} \prod_{t=2}^T A_{z_{t-1}, z_t} \prod_{t=1}^T p(x_t|x_{t-1},z_t) p(y_t^{(c)}|x_t^{(c)}, z_t) p(y_t^{(s)}|x_t^{(s)}, z_t)$$

(Kim et al., 14 Dec 2025)

2. Learning and Inference Algorithms

Expectation-Maximization (EM) for SMDS:

The standard learning strategy is unsupervised EM, maximizing the marginal data log-likelihood $L(\theta) = \log p(y_{1:T}; \theta)$. The complete-data Q-function is:

$$Q(\theta, \theta^{old}) = \mathbb{E}_{p(z, x|y; \theta^{old})}[\log p(z, x, y; \theta)]$$

The E-step employs a forward-backward "switching filter" (e.g., sMSNF) that combines numerical-integration (cubature) approximations for mixed Gaussian and Poisson likelihoods to compute smoothed posteriors over latent regimes and states ($\gamma_{t,k}$, $\xi_{t-1,t,i,j}$, $\hat{x}_{t|T}$, $\Sigma_{t|T}$) (Kim et al., 14 Dec 2025).

The M-step yields closed-form updates for $\pi$, $A$, $F_k^{(m)}$, $Q_k^{(m)}$, $H_k^{(c)}$, $R_k^{(c)}$, and uses a Newton or trust-region solver for the non-conjugate Poisson (spike) observation parameters. Both modalities are fused into the complete-data likelihood, ensuring cross-modal updates (Kim et al., 14 Dec 2025).

Sequential Bayesian inference:

For models with nested nonlinear dynamics and hierarchical multi-temporal structure, posterior inference is performed via Sequential Monte Carlo (SMC) (particle filtering). Fast- and slow-scale states are jointly propagated, resampled, and weighted, with regime switches handled by discrete proposals and filtering at the "coarse" (slow) scale (Vélez-Cruz et al., 24 Oct 2024).

Gibbs sampling with Pólya-Gamma augmentation:

Tree-structured recurrent SLDS employ a hierarchical switching structure and locally linear dynamics. Gibbs sampling is made tractable using Pólya–Gamma augmentation for the logistic path choices at internal tree nodes, yielding efficient updates for all discrete and continuous latent variables (Nassar et al., 2018).

Causation Entropy Boosting (CEBoosting):

For online regime detection and rapid identification of model corrections, CEBoosting computes causation entropy (CSE) at each batch to detect structural changes, accumulates CSE over short batches to stabilize detection, and solves sparse regression problems for post-switch parameter estimation. This approach enables real-time adaptation and regime segmentation in both fully and partially observed systems (Chen et al., 2023).

3. Multiscale and Multimodal Integration

Multiscale hierarchy and nested dynamics:

SMDS models often involve two or more temporal (or spatial) scales, with interactions encoded through nesting (e.g., fast-scale evolution influenced by slow-scale state, and vice versa). Vélez-Cruz & Laubichler (Vélez-Cruz et al., 24 Oct 2024) specify systems where a fast state $x^{F}_{t_1}$ evolves at every "fine" time index, informed by concurrent slow-scale state $x^{S}_{t_2}$, and slow-scale states $x^{S}_{t_2}$ evolve at coarser intervals, integrating summaries over fine-scale trajectories.

Fusion of multiple modalities:

Kim et al. (Kim et al., 14 Dec 2025) designed sMSNF—a recursive filtering strategy that fuses multimodal evidence (Gaussian and Poisson) via fifth-degree cubature integration, weighting contribution by SNR and using a scaling hyperparameter $\tau$ to balance likelihood terms for cross-modality tuning (optimized via downstream decoding). Both modalities influence latent smoothing and M-step updates, enabling robust inference from heterogeneous signals.

Tree-structured switching:

The TrSLDS model establishes a hierarchy wherein discrete state transitions are governed by a binary tree, yielding multi-scale regime partitioning. At each level, leaf nodes correspond to increasingly fine-scale dynamical regimes, and internal nodes capture coarser structure. Dynamics parameters are hierarchically regularized, preserving interpretability and parameter-sharing across scales (Nassar et al., 2018).

4. Regime Detection and Segmentation

SMDS frameworks support both unsupervised segmentation and real-time detection of regime shifts:

• Posterior inference: Soft regime segmentation is obtained via smoothed posterior marginals or maximum-a-posteriori (MAP) state sequences (e.g., via $\gamma_{t,k}$ in EM; via marginal weights in SMC) (Kim et al., 14 Dec 2025, Vélez-Cruz et al., 24 Oct 2024).
• Entropy-based online detection: CEBoosting computes CSE for model residuals against candidate functions in each batch, flagging a regime switch when above-threshold CSE indicators appear and aggregate stably over multiple batches. This avoids premature or noisy detections and is suitable for high-dimensional, intermittent, or heavy-tailed processes (Chen et al., 2023).
• Particle filter smoothing: Bayesian learning schemes maintain regime posteriors via weighted ensembles and identify sharp shifts via thresholding or smoothing of posterior probabilities (Vélez-Cruz et al., 24 Oct 2024).

5. Empirical Results and Performance Metrics

Kim et al. demonstrated that in stationary ($M=1$) scenarios, multiscale models (MSNF-EM) achieve $30\%$–$80\%$ improvements in decoding correlation coefficients and neural self-prediction over single-scale methods, and outperform Laplace-based inference approaches (p < $10^{-5}$). Under switching ($M=2$) regimes, sMSNF-EM achieves regime segmentation accuracy above $90\%$ (versus $70$–$80\%$ for single-scale models), and yields superior field and spike prediction (Kim et al., 14 Dec 2025).

On non-human primate neural data, fusing spike and LFP modalities with sMSNF-EM increases 2D reach decoding CC by $20$–$40\%$ relative to single-scale methods; introducing $M=2$ regime switching improves behavior decoding by a further $10$–$15\%$, consistently outperforming Laplace-based sMSF-EM ($p < 10^{-4}$).

Vélez-Cruz & Laubichler (Vélez-Cruz et al., 24 Oct 2024) validate their SMC-based approach on synthetic multiscale systems with two switching regimes, achieving regime classification error under $5\%$ and coarse-scale RMSE in the $0.11$–$0.16$ range using $N=1000$ particles, and demonstrate scalability and accuracy across individuals.

TrSLDS is empirically validated on oscillatory dynamics, chaotic attractors (e.g., FitzHugh–Nagumo, Lorenz), and macaque V1 neural data, achieving multi-scale vector-field recoveries and predictive R$^2$ superior to SLDS and rSLDS, with interpretable latent regime structure (Nassar et al., 2018).

CEBoosting demonstrates rapid regime detection (≈2 decorrelation times for Lorenz-96 system), resilience to partial observation, and accurate recovery of new regime parameters in the presence of intermittency and high-dimensionality (Chen et al., 2023).

6. Applications and Extensions

Neural systems and brain–computer interfaces:

SMDS models robustly track non-stationary and regime-dependent neural dynamics, supporting stable behavior decoding in the face of neural tuning drift or modality degradation. Behavioral regime segmentation reveals latent network states and neuromodulatory influences (Kim et al., 14 Dec 2025).

Complex system analysis:

Vélez-Cruz & Laubichler's framework generalizes to diverse domains with nested multiscale dynamics, enabling identification of both transient and persistent regime shifts, with particle filtering accommodating nonlinear and non-Gaussian settings (Vélez-Cruz et al., 24 Oct 2024).

Online detection and real-time adaptation:

CEBoosting provides lightweight, online regime segmentation and parameter rectification via sequential causation entropy analysis and sparse regression, with application to meteorological, geophysical, and high-dimensional neural systems (Chen et al., 2023).

Model extensions:

SMDS approaches have been extended to accommodate more than two regimes ($M>2$), hierarchical or semi-Markov regime processes, modality-specific latent embeddings, nonlinear dynamics (Gaussian processes/deep nets), and online/stochastic-variational adaptation (Kim et al., 14 Dec 2025, Vélez-Cruz et al., 24 Oct 2024, Nassar et al., 2018).

7. Interpretability, Scalability, and Limitations

Interpretability vs. complexity:

Tree-structured approaches (TrSLDS) allow controlled tradeoff between interpretability and granularity through truncation at varying depths. Hierarchical priors control overfitting and promote parameter sharing across scales (Nassar et al., 2018).

Scalability:

Particle filtering and online CSE approaches retain scalability for moderately large-dimensional systems via localization (restriction of candidate libraries) and efficient computation of low-rank covariances (Chen et al., 2023, Vélez-Cruz et al., 24 Oct 2024).

Limitations and future directions:

• Gaussian approximations in CSE may underperform when faced with extremely non-Gaussian or delayed couplings.
• Thresholds, batch sizing, and smoothing parameters must be tuned relative to autocorrelation scales.
• Model extensions to fully nonlinear, deep, or semi-Markov structures are active research topics, as is the enforcement of physics-based constraints (e.g., conservation laws) (Chen et al., 2023, Kim et al., 14 Dec 2025).

In summary, SMDS models constitute a mathematically principled and computationally efficient framework for capturing, segmenting, and decoding regime-switching, multiscale dynamical processes in complex systems. Their modular design, flexible fusion of modalities, and robust inference algorithms establish them as a foundational tool in time-series analysis across neuroscience, geoscience, and engineered systems (Kim et al., 14 Dec 2025, Vélez-Cruz et al., 24 Oct 2024, Nassar et al., 2018, Chen et al., 2023).