Dynamic Gaussian Processes and the Vanilla-SPDE Exchange

Abstract: Gaussian process inference is often limited by cubic computational costs, a challenge that becomes more pronounced in spatio-temporal settings where posterior inference is required over dense grids. While state-space SPDE formulations enable linear complexity in time, exact inference remains cubic in space and deteriorates further when observation locations are disjoint from the prediction locations, which inflates the number of considered spatial points. To address this, we propose the Vanilla-SPDE Exchange, which exploits an equivalence between the standard and SPDE formulations of GP inference to construct a hybrid scheme with improved computational cost. We demonstrate these gains through complexity analysis and numerical experiments.

• The paper introduces the Vanilla-SPDE Exchange (VaSE), a hybrid method that combines static GP regression with dynamic SPDE sampling to overcome computational bottlenecks.
• It demonstrates that VaSE achieves lower runtime and comparable memory usage, particularly when observation and prediction locations are disjoint.
• The study provides rigorous complexity analysis and empirical benchmarks, expanding the applicability of GP models for efficient high-resolution spatio-temporal forecasting.

Dynamic Gaussian Processes and the Vanilla-SPDE Exchange: A Technical Analysis

Introduction

Gaussian Process (GP) models are foundational in statistical modeling of spatio-temporal data, offering non-parametric expressive power and rigorous quantification of uncertainty. However, the cubic computational cost associated with standard GP inference remains a principal limitation, particularly acute in high-resolution spatio-temporal domains where inference on dense grids is required. The state-space framework via Stochastic Partial Differential Equations (SPDEs) allows for linear-time inference in the temporal dimension, but exact inference generally retains cubic scaling in the spatial dimension. This issue becomes particularly salient in settings where observation and prediction locations are spatially disjoint, causing the effective spatial dimension to balloon. The paper "Dynamic Gaussian Processes and the Vanilla-SPDE Exchange" (2606.31063) addresses these computational bottlenecks by introducing the Vanilla-SPDE Exchange (VaSE), a hybrid inference technique that fuses the computational strengths of both standard and SPDE-based GP inference.

Static and Dynamic Gaussian Process Inference Methods

The conventional, or static, GP formulation constructs a joint covariance over all observation and prediction locations, treating the process as a large finite-dimensional multivariate Gaussian. The primary computational cost arises from the inversion and Cholesky decomposition of dense covariance matrices, with complexity $O(N^3)$ for $N$ total points. Prediction over spatio-temporal grids further exacerbates this cost, as both spatial ($M_s$) and temporal ($M_t$) grid sizes interact multiplicatively.

Dynamic inference reformulates a subclass of GPs—especially those with Matérn kernel in time and separable spatial-temporal structure—as solutions to SPDEs. This recasting enables sequential state-space inference using Kalman filtering and Rauch–Tung–Striebel (RTS) smoothing. For temporal Matérn kernels with half-integer order, exact Markovian embeddings in the state-space are available, permitting linear scaling in the number of time steps. However, when the set of considered spatial locations is large or non-overlapping between observations and predictions, the cost per step is $O(N_s^3)$, with $N_s$ the number of spatial locations, resulting in overall complexity $O(N_s^3 N_t)$.

The paper offers a rigorous complexity comparison between the standard (static) and dynamic (SPDE-based) GP inference across scenarios, demonstrating the non-uniform dominance of either approach depending on data geometry—particularly the alignment or disjointness of spatial observation and prediction sets.

Figure 1: Illustration of overlapping (top) and disjoint (bottom) observation–prediction locations in one-dimensional space over time.

The Vanilla-SPDE Exchange (VaSE)

The VaSE procedure exploits the equivalence between kernel-based and SPDE-based GP formulations, allowing a switch between static and dynamic views within a single inference pipeline. The highlight is the scenario where observation and prediction locations are substantially disjoint. Here, direct SPDE inference is suboptimal because the state must encompass the union of observation and prediction sites, inflating the spatial dimension and the cost of every Kalman update.

The core of VaSE is as follows:

• Regression is performed using the static GP (kernel-based) formulation, potentially with derivatively-augmented kernels to recover the state-space sufficient statistics at the prediction grid and the forecast boundary.
• The posterior mean and covariance for the SPDE state at the prediction sites and final observation time are extracted from this kernel-based posterior.
• Forecasting and posterior sampling for times beyond observed data are performed dynamically using the SPDE state-space equations starting from this initial posterior, leveraging linear-in-time propagation.

This hybrid approach maintains the computational tractability of the static method for the regression phase and the efficiency of the SPDE propagation for posterior sampling, notably reducing the regression bottleneck specific to disjoint settings.

Computational and Empirical Evaluation

Theoretical complexity analysis demonstrates that, under the VaSE strategy, the regression cost is $O(N^3 + N^2 M_s + N M_s^2)$ for $N$ observations and $M_s$ prediction locations, and posterior sampling for $N$0 instances across a grid of size $N$1 requires $N$2 operations. In comparison, vanilla kernel-based methods scale at least as $N$3 in the sampling phase, while SPDE-based approaches are penalized by $N$4 regression cost with $N$5 the union of all spatial sites.

Empirical benchmarks support this analysis, particularly in the regime of disjoint observation–prediction locations, where VaSE exhibits superior runtime and memory profiles over both pure static and pure dynamic approaches.

Figure 2: Empirical runtime and peak memory usage for posterior sampling versus number of observations, comparing vanilla, SPDE, and VaSE methods for both overlapping and disjoint scenarios.

The experimental setup uses a two-dimensional spatial RBF kernel crossed with a temporal Matérn-3/2 kernel, with synthetic observations and required posterior sampling on dense spatio-temporal grids. VaSE achieves lower runtime and comparable memory consumption to SPDE, especially as the number of disjoint observations increases, validating the scalability analysis.

Implications and Potential Extensions

VaSE presents significant practical utility for spatio-temporal inference tasks in environmental science, remote sensing, and any application requiring rapid forecasting or sampling conditioned on historical data collected at arbitrary or mobile locations. Its hybrid strategy sidesteps the core inefficiency of existing approaches in scenarios where observation-prediction location overlap is minimal.

Theoretically, VaSE reifies the operational equivalence of GP formulations and linear state-space models under differentiation and linear operator augmentation, an aspect ripe for further generalization, including to non-separable and non-stationary kernels via approximate embeddings or spectral representations.

Potential future developments could include:

• Extension to non-separable or non-Matérn kernels through approximate Markov embeddings [see, e.g., "A general linear-time inference method for Gaussian Processes on one dimension" (Loper et al., 2021)], potentially by spectral or Kronecker-based decompositions.
• Integration with sparse or low-rank approximation techniques, further accelerating the static regression step without compromising inference accuracy [cf. variational approaches and Vecchia approximations].
• Automating the decomposition and exchange within generic probabilistic programming or scalable Bayesian inference pipelines.
• Application to real-world datasets where spatial coverage shifts dynamically, such as mobile sensor networks or active learning for environmental monitoring.

Conclusion

The Vanilla-SPDE Exchange paradigm synthesizes the computational advantages of both standard (kernel-based) and SPDE-based (dynamic, state-space) GP inference. It provides an exact inference framework that is adaptively efficient across a range of spatio-temporal geometries, particularly excelling when observation and prediction locations are disjoint. The method is supported by rigorous complexity analysis and empirical validation. VaSE induces a significant expansion of the practical scope of GP models for high-resolution spatio-temporal inference and suggests a broader class of hybrid-Gaussian inference methods to be explored in future research.