---
title: State-Space Representation of GP Priors
url: https://www.emergentmind.com/topics/state-space-representation-of-gaussian-process-priors
type: topic
---

# State-Space Representation of GP Priors

A state-space representation of a Gaussian process (GP) prior is a linear Markovian dynamical system—usually formalized as a stochastic differential equation (SDE), or, after temporal/spatial discretization, a vector autoregressive process—whose output process shares the covariance structure of the original GP. For stationary GP priors with rational spectral densities, the Markovian property can be made exact in finite (typically low) state dimension, yielding a canonical route to O(n) inference via Kalman filtering. State-space representations extend beyond stationary GPs to a wide class of nonstationary priors, spatio-temporal models, and deep hierarchical constructions, often with significant benefits in interpretability and computational efficiency.

## 1. Fundamentals of State-Space Representations for Gaussian Processes

State-space representations of GPs recast a stochastic process prior as a latent Markov process observed through linear measurements corrupted by Gaussian noise. The canonical continuous-time linear SDE form is:
\[
\frac{d}{dt} f(t) = F(t) f(t) + L(t) w(t)
\]
with state vector $f(t) \in \mathbb{R}^m$, time-dependent drift $F(t) \in \mathbb{R}^{m\times m}$, diffusion $L(t) \in \mathbb{R}^{m \times 1}$, and driving white noise process $w(t)$ of variance $q(t) \geq 0$. The corresponding observation at time $t_k$ is
\[
y_k = H_k f(t_k) + \varepsilon_k
\]
where $H_k = H(t_k)$, and $\varepsilon_k \sim \mathcal{N}(0, \sigma_v^2)$ [1601.01544].

This formulation fully characterizes the joint distribution over arbitrary measurement sets through the solution to the linear SDE, which leads directly to closed-form expressions for the predictive mean and covariance [2001.05363].

For GPs with stationary, rational spectral densities, such as Matérn kernels with half-integer smoothness parameter $\nu = p+1/2$, the state-space form is exact and minimal. For example, the Matérn-1/2 (OU) process is represented with a 1D state, Matérn-3/2 with a 2D state, and Matérn-5/2 with a 3D state, with explicit formulae for $(F, L, H)$ as a function of the kernel hyperparameters [2511.18797, 2111.12604].

## 2. Kernel Classes and Explicit State-Space Mappings

The key property enabling finite-dimensional state-space conversion is that the GP prior's spectral density $S(\omega)$ is rational in $\omega^2$. An explicit factorization $S(\omega)=G(i\omega)G(-i\omega)^*$ admits a transfer function realization
\[
G(s) = H(sI - F)^{-1} L
\]
so that the process output exactly matches the original GP covariance [2008.04733].

Key kernel classes with explicit state-space mappings include:
- **Matérn kernels with half-integer $\nu$**: finite-dimensional, minimal representations; see explicit matrices for $\nu = 1/2, 3/2, 5/2$ in [2001.05363, 2511.18797, 2111.12604].
- **Damped simple harmonic oscillator (DSHO) kernels**: 2D state representing underdamped oscillatory GPs [2109.10685].
- **Polynomial and cubic spline kernels**: as degenerate cases with deterministic SDEs or simple driving white noise [1601.01544].
- **Periodic and neural network kernels**: using block-diagonal or time-varying (LTV/LTI) structures for the drift and observation matrices [1601.01544].

Nonstationary or non-rational-spectral-density kernels (e.g., the squared-exponential) admit only approximate finite state-space realizations via truncation, Taylor expansion, or spectral projection [1601.01544, 1506.02267].

## 3. Computational Implications and Inference Algorithms

State-space GPs provide a profound computational advantage. Discretizing the SDE yields a state-evolution recursion:
\[
x_{k+1} = A_k x_k + w_k
\]
\[
y_k = H_k x_k + \varepsilon_k
\]
with $A_k = \exp(F \Delta t)$ and process noise covariance $Q_k$ computed exactly via matrix integrals. For time-invariant $F, L$, $A$ and $Q$ can be precomputed [2001.05363].

Inference (filtering and smoothing) is performed via Kalman filter and Rauch-Tung-Striebel (RTS) smoother recursions, with per-step complexity $O(m^3)$ and total complexity $O(n m^3)$ (often $O(n m^2)$ for sparse operations), contrasted with the $O(n^3)$ scaling for standard GP inference [2511.18797, 2111.12604].

The state-space formulation is amenable to modern scalable variational inference via block-banded precisions (enabling “doubly sparse” approximations), sparse-inducing points, and efficient natural gradient updates [2001.05363].

## 4. Extensions: Nonstationarity, Spatio-Temporal, and Basis Expansions

Nonstationary GP priors are addressed by embedding transient terms (effect of initial state) and explicit time-varying matrices $F(t), L(t), H(t)$, and allowing for nontrivial initial covariance $\Sigma_0$ [1601.01544]. Canonical examples include polynomial regression priors, periodic kernels (via sums of cos/sin modes), and time-localized Gaussian kernels. For the squared exponential, rational function approximation of the inverse spectral density enables approximate finite-dimensional realization, with explicit error quantification [1601.01544, 1506.02267].

Spatio-temporal GPs with convolution-generated or SPDE-based priors are formalized through infinite-dimensional SDEs. Galerkin projection onto finite orthonormal spatial bases yields a large but finite-dimensional linear state-space system, whose process noise statistics and operator structure match the original GP at the respective approximation level. The covariance error decays as the tail of the Karhunen-Loève spectrum of the kernel [2512.01279, 1506.02267].

Basis-expansion models (e.g., truncated Mercer or Karhunen-Loève), where the GP is projected onto the span of leading eigenfunctions, provide an interpretable and flexible yet computable state-space surrogate. The induced prior on basis coefficients is Gaussian, matching the kernel's spectral density at corresponding eigenvalues [1506.02267, 1603.05486].

| Kernel Type                 | State Dimension | State-Space Construction                         |
|-----------------------------|----------------|--------------------------------------------------|
| Matérn-$\frac{1}{2}$        | 1              | 1D OU process, LTI SDE                           |
| Matérn-$\frac{3}{2}$        | 2              | 2D LTI SDE, see explicit $F$, $L$ in [2001.05363]|
| Matérn-$\frac{5}{2}$        | 3              | 3D LTI SDE, higher-order derivatives             |
| SE/Exp. rational approx     | $d$ (var.)     | Taylor-truncated LTI                             |
| DSHO/Periodic               | 2 (per freq.)  | Block-diagonal LTI                               |
| Infinite-dimensional (SPDE) | $\infty$       | SDE projected via Galerkin (truncated to $N$)    |

## 5. Hierarchical and Deep Gaussian Processes in State Space

State-space methods extend to deep and hierarchical models by stacking GP priors whose hyperparameters (length-scales, amplitudes) are themselves outputs of other state-space GPs. This yields a nonlinear, hierarchical SDE system, where each layer depends on its parents’ realization, and the overall composite is again Markovian in a high-dimensional state [2008.04733, 2111.12604].

Inference for deep state-space GPs leverages the Markovian structure, employing extended/cubature Kalman filters or particle methods as needed for nonlinear transitions. This approach allows for efficient O(n) smoothing in models intractable to classical DGP inference.

## 6. Practical Considerations and Limitations

The state-space approach relies on the existence or tractability of the SDE realization for the target kernel. For Matérn kernels and close rational-spectral-density variants, this yields practical finite-dimensional systems. For kernels without a rational factorization (e.g., squared exponential), only approximate state-space representations are available, often with a trade-off between approximation error and computational cost [1601.01544].

Truncated basis projections or Galerkin methods introduce finite-dimensional error that decays with the number of basis terms. Choice of basis (Fourier, Laplace, finite element) impacts computational efficiency, especially for complex geometries in space [2512.01279, 1506.02267].

All state-space systems constructed this way are linear-Gaussian, making the approach most natural for priors and likelihoods compatible with these assumptions. Highly nonlinear or non-Gaussian extensions require additional approximation.

## 7. Applications and Impact

State-space representations of GP priors have substantially extended the range of GPs amenable to large-scale inference, especially for temporal, spatio-temporal, and hierarchical problems in fields such as time series analysis, control, signal processing, and spatiotemporal environmental modelling [2109.10685, 2512.01279]. The ability to exploit the Markov property and block-banded precision structures yields scalable algorithms for streaming, real-time, and massive-data scenarios, while offering a link to physically meaningful dynamical models [2001.05363]. These techniques underpin a new generation of efficient, flexible, and interpretable probabilistic modelling frameworks.

Source: https://www.emergentmind.com/topics/state-space-representation-of-gaussian-process-priors