Infinite-Dimensional Gaussian Processes

• Infinite-dimensional Gaussian processes are stochastic models defined on function spaces with Gaussian measures and trace-class covariance operators.
• They bridge advanced frameworks, linking RKHS, optimal transport geometry, and functional regression to support scalable uncertainty quantification.
• Applications include operator learning in stochastic PDEs, Bayesian inversion, and adaptive Monte Carlo methods for high-dimensional inference.

An infinite-dimensional Gaussian process is a probabilistic object whose sample paths, parameterized by time, space, or more general indices, take values in infinite-dimensional function spaces such as separable Hilbert or Banach spaces. Such processes are characterized by laws that are Gaussian measures on these topological vector spaces, equipped with covariance operators of trace-class. Infinite-dimensional Gaussian processes are foundational in stochastic analysis, statistical learning theory, Bayesian field inference, operator learning, and high-dimensional uncertainty quantification. The mathematical and algorithmic infrastructure for their study includes measure-theoretic constructions, operator theory, stochastic PDEs, infinite-dimensional statistical geometry, optimal transport, and numerical methods for function-aligned inference.

1. Infinite-Dimensional Gaussian Measure Framework

An infinite-dimensional Gaussian process is defined as a collection $\{X_t\}_{t \in D}$ (or, more generally, random elements $X$ in a separable Hilbert space $H$) such that every finite linear combination of the coordinates is (real) Gaussian $N(0,\sigma^2)$. The law of $X$ can be identified with a Borel Gaussian measure $\mu \in \mathcal{P}_2(H)$, specified by a mean $m \in H$ and a positive, self-adjoint, trace-class covariance operator $C : H \to H$ (Quang, 2020, Wallin et al., 2018). For $\ell_1,\ell_2 \in H^*$, the covariance is

$$\langle u, C v \rangle = \int_H \langle x-m, u \rangle \langle x-m, v \rangle \, d\mu(x)$$

and induces a Cameron–Martin space $E = \text{Range}(C^{1/2})$ with inner product $$\langle u, v \rangle_E = \langle C^{-1/2}u, C^{-1/2}v \rangle$$. The Karhunen–Loève expansion is

$$X = m + \sum_{k=1}^\infty \sqrt{\lambda_k}\, \xi_k\, e_k$$

with $\lambda_k, e_k$ the eigenpairs of $C$ and $\xi_k \sim N(0,1)$ independent (Wallin et al., 2018). Establishing this infinite-dimensional foundation is essential for advanced modeling of function-valued phenomena, nonparametric Bayes, adaptive Monte Carlo algorithms, and operator inference.

2. Covariance Structure and RKHS Connections

The covariance kernel $K(x, y) = \mathbb{E}[X_x X_y]$ associated to a Gaussian process induces a reproducing kernel Hilbert space (RKHS) $\mathcal{H}_K$ on the index set $D$ (Jorgensen et al., 2022). Given any positive-definite kernel $K : D \times D \to \mathbb{R}$, there exists a unique, centered Gaussian process $X$ whose $L^2$ closure of finite linear combinations of $X_{t}$ is isomorphic to $\mathcal{H}_K$ (Jorgensen et al., 2022). The Cameron–Martin space of the Gaussian measure coincides with the RKHS, and point evaluation is continuous. Isomorphism theorems exist between measure spaces of finite (signed) energy and $\mathcal{H}_K$ via $T_K \mu = \int K(s, \cdot)\, d\mu(s)$, and between $L^2$ path spaces and $\mathcal{H}_K$ (infinite-dimensional Fourier transform) via $\mathcal{T} F(t) = \mathbb{E}[e^{-i X_t} F]$ (Jorgensen et al., 2022), enabling spectral and pathwise analysis on stochastic function spaces.

3. Conditioning, Disintegration, and Infinite-Dimensional Regression

Given a Gaussian measure $\mu$ on a Banach space $X$ (e.g., $C(D)$), linear observations $y = G(x)$ for bounded operators $G : X \to Y$ admit an explicit conditional Gaussian measure, or disintegration, provided the pushforward covariance $C_\nu = G C_\mu G^*$ is of finite rank (2207.13581, LaGatta, 2010). The conditional mean is affine in $y$, and the conditional covariance is independent of $y$. This yields consistent update rules for Gaussian process regression even when the observation operators are infinite-dimensional or involve linear functionals, as in functional data analysis or PDE-constrained Bayesian inversion: $$m_1 = m_0 + C_0\, G^* (G\, C_0\, G^*)^{-1}(y - G m_0), \quad C_1 = C_0 - C_0\, G^* (G\, C_0\, G^*)^{-1} G C_0$$ (2207.13581). These equations generalize the Kalman filter and Bayesian posterior update to Hilbert spaces. Continuous disintegration is ensured precisely when a certain operator norm $M$ is finite (LaGatta, 2010).

4. Entropic Regularization and Optimal Transport Geometry

Entropic optimal transport on infinite-dimensional Hilbert spaces provides a differentiable interpolation between squared $L^2$-distance and classical quadratic Wasserstein distance, addressing the singularities and ill-posedness of the exact $W_2$ metric in infinite dimensions (Quang, 2020). For Gaussian measures $\mu_0 = N(m_0, C_0)$ and $\mu_1 = N(m_1, C_1)$, the entropic-regularized cost is

$$\mathrm{OT}_{2}^\epsilon (\mu_0, \mu_1) = \inf_{\gamma} \mathbb{E}_\gamma \|x-y\|^2 + \epsilon \, \mathrm{KL}(\gamma \| \mu_0 \otimes \mu_1)$$

whose unique minimizer is a joint Gaussian coupling with explicit cross-covariance, and a closed-form formula for the cost in terms of the means and covariances. The associated Sinkhorn divergence $S_2^\epsilon$ is strictly positive definite and Fréchet differentiable everywhere, in contrast to $W_2$ (Quang, 2020).

For sets of Gaussian measures (e.g., for Bayesian model averaging or dictionary learning in function space), the Sinkhorn barycenter has a unique solution in Hilbert spaces, given by a matrix fixed-point equation involving the input covariances (Quang, 2020). In the special case of Gaussian measures on RKHSs, the entropic-regularized OT and Sinkhorn divergences admit explicit kernel matrix expressions, interpolating between the kernel MMD and the Wasserstein metric.

5. Construction and Analysis of Kernels in Distribution Spaces

To index GPs on probability distributions, optimal transport-based Hilbertian embeddings have been developed (Bachoc et al., 2018). Measures $\mu$ are mapped to $L^2(\bar{\mu})$ via optimal transport to a Wasserstein barycenter, linearizing the Wasserstein geometry for kernel construction. Radial kernels $F(\|\Phi(\mu) - \Phi(\nu)\|)$ where $F$ is completely monotone generate positive definite kernels on $L^2$-spaces, ensuring strict positive definiteness, rotational invariance, and microergodicity (every distinct parameter induces singular laws in infinite dimensions). Consistency holds for empirical barycenters and such approaches enable the extension of regression techniques to spaces of distributions (Bachoc et al., 2018).

6. Operator-Valued and Pathwise Constructions

Infinite-dimensional Gaussian processes arise as solutions of infinite-dimensional stochastic differential equations (SDEs), including SPDEs and Volterra-type integrals driven by Hilbert-valued Gaussian noise (Benth et al., 2020, Beghin et al., 2023). The covariance of such pathwise constructions can be analyzed using 2D Volterra sewing lemmas, yielding operator-valued covariance functionals. Fractional, Ornstein–Uhlenbeck, and rough path extensions are naturally accommodated. Generalized fractional operators parameterized by Bernstein functions permit modeling of long- and short-range dependence, persistence, and local time properties (Beghin et al., 2023). Regularity, entropy, and the spectrum of these covariances are central in both the existence of rough-path lifts and the design of MCMC algorithms for functional Bayesian inference (Wallin et al., 2018).

7. Applications, Computation, and Inductive Bias

Infinite-dimensional Gaussian processes underpin kernel methods for functional data, stochastic PDE modeling, operator learning, and uncertainty quantification in distributed systems (Souza et al., 19 Oct 2025, Courteville et al., 15 Dec 2025). Implementations crucially exploit discretized representers from the underlying Hilbert or RKHS structure, adaptive MCMC in function spaces, and the differentiability properties of regularized transport for high-dimensional optimization. Neural operator limits, tensor network GP analogues, and port-Hamiltonian system surrogates demonstrate the breadth of inductive biases and architectures that can be embedded within principled infinite-dimensional GP frameworks (Souza et al., 19 Oct 2025, Guo et al., 2021, Courteville et al., 15 Dec 2025).

In summary, infinite-dimensional Gaussian processes unify advanced probabilistic modeling on function spaces, computational methods in high- or infinite-dimensional regimes, regularized transport geometry, and functional operator learning, with substantial theoretical infrastructure supporting practical, scalable, and well-posed inference (Quang, 2020, Bachoc et al., 2018, Beghin et al., 2023, Benth et al., 2020, Jorgensen et al., 2022, Souza et al., 19 Oct 2025, Courteville et al., 15 Dec 2025).