Graph State Space Models

• Graph State Space Models are statistical frameworks that combine state space models with graph-structured dependencies to capture interactions among latent variables.
• They employ sparse polynomial and linear parameterizations to recover both dynamic interactions and underlying graph topology through regularized likelihood optimization.
• These models achieve superior recovery and interpretability in complex systems by leveraging scalable inference methods such as proximal gradient and EM algorithms.

A Graph State Space Model (GSSM) is a statistical modeling framework that combines classic state space models with graph-structured dependencies among latent system variables. In GSSMs, the evolution of the latent state is explicitly governed by a (typically sparse) interaction graph encoding which components influence each other, supporting both linear and nonlinear dynamics. This formalism enables modelers to impose, infer, or exploit structured dependencies in large-scale or high-dimensional dynamical systems, offering performance and interpretability benefits over models ignoring graph structure (Cox et al., 2024).

1. Mathematical Formulation and Core Principles

A GSSM formalizes the evolution of a latent state vector $x_t \in \mathbb{R}^n$ such that

$$x_t = f(x_{t-1}) + \eta_t,\quad \eta_t \sim Q(\cdot),$$

with observations

$$y_t = g(x_t) + \epsilon_t,\quad \epsilon_t \sim R(\cdot).$$

Crucially, the transition map $f$ and (often) the observation map $g$ are constrained by or parameterized through an adjacency matrix $A \in \{0,1\}^{n\times n}$, where $A_{ij} = 1$ denotes that component $j$ directly influences component $i$.

In GraphGrad, for nonlinear systems, $f$ is parameterized as a sparse polynomial: $$f(x) \approx \sum_{|\alpha|\le p} C_\alpha \odot x^\alpha,$$ where $\alpha$ indexes multi-degrees and $C_\alpha \in \mathbb{R}^n$; the sparsity pattern of $\bigl\{ C_\alpha \bigr\}$ directly encodes the graph $A$. Influence from $j$ to $i$ is present when some $\alpha$ with $\alpha_j > 0$ and $C_{i,\alpha} \neq 0$ exists (Cox et al., 2024). For linear-Gaussian GSSMs, the transition matrix $A$ itself serves as the (weighted, directed) adjacency, and sparsity reflects Granger-causal structure among state components (Chouzenoux et al., 2023, Elvira et al., 2022).

2. Sparse Polynomial and Linear Parameterizations

Approaches to parameterizing $f$ and $A$ in GSSMs:

• Polynomial Expansion (GraphGrad): The transition $f(x)$ is approximated as a sum over monomials $x^\alpha$ up to total degree $p$, with coefficient vectors $C_\alpha$ encoding edge structure. This allows recovery of both graph topology $A$ and nonlinear interaction strengths from observed data via $\ell_1$-type sparse regularization (Cox et al., 2024).
• Linear State Transition: In linear-Gaussian settings, the state dynamics are $x_k = A x_{k-1} + w_k$, $y_k = C x_k + v_k$, with $A$ sparse and interpreted as an adjacency encoding direct Granger-causal links. Various works, including GraphIT and GraphEM, leverage sparsity-promoting priors (e.g., $\ell_1$, non-convex surrogates) and EM-type algorithms to estimate $A$ under model and data constraints (Chouzenoux et al., 2023, Elvira et al., 2022).

These parameterizations support estimation of both the graph structure and system dynamics, addressing identifiability and interpretability.

3. Inference and Optimization Techniques

Estimation of GSSM parameters typically relies on penalized likelihood optimization:

• Marginal Likelihood for Nonlinear GSSMs: With polynomial $f$, the negative log-likelihood is approximated by differentiable particle filtering followed by sparsity-penalized optimization:

$$\ell(\theta) = -\log L(\theta) + \lambda\,\Omega(\theta),\quad\Omega(\theta) = \sum_{|\alpha|\le p}\|C_\alpha\|_1.$$

Stochastic proximal-gradient methods with soft-thresholding yield both sparse and stable solutions (Cox et al., 2024).

• Majorization-Minimization and Proximal Splitting for Linear GSSMs: Both GraphIT and GraphEM develop iterative convex surrogates for the penalized negative log-likelihood, solving at each iteration a convex optimization problem via proximal splitting (e.g., Douglas–Rachford) for $\ell_1$ or more sophisticated penalties, using EM to alternate between inference and parameter update (Chouzenoux et al., 2023, Elvira et al., 2022).

These frameworks allow for efficient and stable parameter recovery in both dense and highly sparse regimes.

4. Algorithmic Workflows and Computational Aspects

The general optimization workflow in GSSMs includes:

Stage	Approach (Nonlinear)	Approach (Linear)
State/Obs. Simulation	Monte Carlo Particle Filter	Kalman Smoother
Objective Eval.	Monte Carlo Likelihood	EM Surrogate
Regularization	$\ell_1$ (proximal) penalty	$\ell_1$/group/Laplace penalty
Optimization	Stochastic Proximal Gradient	Proximal Splitting (DR/etc.)

For nonlinear polynomial GSSMs, batching and mini-batch optimization (Batched-GraphGrad) are deployed to address likelihood degeneracy in long sequences, while maintaining overall computational efficiency—forward/backward costs are $O(B S K T)$ (batches, steps, particles, time) (Cox et al., 2024).

For linear-Gaussian cases, each EM outer-loop involves a Kalman smoother (to generate sufficient statistics) and an inner convex optimization (with per-iteration cost dominated by matrix operations or soft-thresholding), allowing applicability to high-dimensional systems (Chouzenoux et al., 2023, Elvira et al., 2022).

5. Empirical Performance and Recovery Guarantees

Empirical validations span both chaotic nonlinear systems and high-dimensional stochastic linear systems:

• GraphGrad recovers exact support (precision=recall=1.0) and achieves near-oracle RMSE on Lorenz-63 and Lorenz-96 (e.g., for $n=20$, $p=2/3$, perfect $F_1$ for $T \geq 200$, RMSE $\sim 3 \times 10^{-4}$), outperforming dense MLE and naive subgradient approaches (Cox et al., 2024).
• GraphIT/GraphEM accurately reconstruct the true transition matrices and edge supports in synthetic LG-SSMs, with F1-scores and detection accuracy markedly superior to unconstrained EM or naive Granger-causality methods, especially in highly sparse or structured regimes (Chouzenoux et al., 2023, Elvira et al., 2022).

Penalty selection and initialization are critical: proper use of $\ell_1$/nonconvex surrogates, majorization-minimization, and variational inference are all empirically necessary for accurate and robust recovery.

6. Extensions, Limitations, and Open Directions

Current GSSM approaches address both linear and nonlinear, directed and undirected (or functional) graphs, but each framework exhibits limits:

• Assumptions: Gaussian noise, separable transition structure, and stationarity are common constraints.
• Extensions: Future directions include error-bound analysis (comparing EKF, Unscented KF, GSSM), automatic identification of constant vs. dynamic variables, non-Gaussian robustness, and scalable real-time inference via incremental solvers or factor graphs (Lü, 2021).
• Limitations: Loopy, multi-connected factor-graph formulations require careful inference (junction tree, window size), and performance may degrade with ill-posed parameterizations or insufficient data.

Ongoing research in GSSMs targets more general dynamics, dynamic graphs, heterogeneous data, and principled methods for structure learning, exploiting advances in sparse optimization and scalable inference (Cox et al., 2024, Chouzenoux et al., 2023, Elvira et al., 2022).

7. Relationship to Broader SSM and Graph Learning Literature

GSSMs generalize classic state space models—Kalman filters, hidden Markov models, polynomial autoregressive models—by incorporating explicit, learnable, or prior-specified graph structure into the dynamics. This positions them at the intersection of dynamical systems, probabilistic graphical models, and recent graph neural network research. Techniques such as particle filtering, proximal-gradient, EM, and variational inference form the algorithmic backbone of modern GSSM estimation (Cox et al., 2024, Chouzenoux et al., 2023, Elvira et al., 2022). The learned or imposed graph structure provides interpretable insights into the system's dynamics and causal structure—an advantage central to applications in neuroscience, multivariate forecasting, physics, and network science.