Gaussian Process Latent Force Model (GPLFM)

• GPLFM is a hybrid framework that combines physical differential equations with nonparametric Gaussian process priors to flexibly model unknown inputs.
• Its additive formulation allows closed-form Bayesian inference while the multiplicative extension accounts for geometric constraints in system dynamics.
• Approximate methods like Picard iteration are used to tackle the computational challenges in the multiplicative case, balancing model fidelity and tractability.

A Gaussian Process Latent Force Model (GPLFM) is a hybrid modeling framework that combines mechanistic equations—typically formulated as ordinary or partial differential equations for physical systems—with nonparametric Gaussian process (GP) priors over unknown or latent input functions. GPLFM provides a Bayesian approach for learning and inference in dynamical systems where the full mechanistic specification may be impractical or underconstrained, allowing the flexible modeling of unknown inputs while maintaining a physically interpretable structure. Unlike purely mechanistic or purely data-driven models, GPLFMs offer an optimal trade-off between model fidelity and flexibility, supporting applications in system identification, control, and prediction across physical, engineering, and biological domains (Tait et al., 2018).

1. Mathematical Formulation and Linear Additive GPLFM

The standard additive GPLFM is built upon a linear state-space evolution equation for a $K$-dimensional state $x(t)\in\mathbb{R}^{K}$: $\dot{x}(t) = A\,x(t) + B\,f(t)$ where $A$ is a $K\times K$ system matrix, $B$ is a $K\times R$ input matrix, and $f(t)\in\mathbb{R}^{R}$ is an unknown latent forcing function. Each component $f_r(t)$ is modeled as an independent GP: $f_r(t) \sim \mathcal{GP}(0, k_r(t, t'))$ The explicit solution for the state is

$$x(t) = \Phi(t)\,x(0) + \int_0^t \Phi(t-\tau)\,B\,f(\tau)\,d\tau$$

with fundamental matrix $\Phi(t)=\exp[A t]$. This construction ensures that $x(t)$ is an affine mapping of $f(\cdot)$ and thus itself a GP: $$\mathrm{Cov}[x(t), x(t')] = \Phi(t)\,\mathrm{Cov}[x(0)]\,\Phi(t')^{\top} + \int_0^t\!\!\int_0^{t'} \Phi(t-\tau)\,B\,k(\tau, \tau')\,B^{\top}\,{\Phi(t'-\tau')}^{\top}d\tau\,d\tau'$$ The resulting model allows closed-form marginal likelihoods and supports efficient GP regression, hyperparameter optimization, and posterior smoothing (Tait et al., 2018).

2. Multiplicative Extension: The Multiplicative Latent Force Model (MLFM)

The MLFM generalizes the additive GPLFM by introducing multiplicative coupling between the latent GP and the system state: $$\dot{x}(t) = A(t)\,x(t),\quad A(t) = A_0 + \sum_{r=1}^R A_r\,g_r(t)$$ where each $g_r(t)$ is an independent GP and $A_r$ are fixed matrices. This induces a matrix-valued stochastic process for $A(t)$. If $\{A_r\}$ are elements of a Lie algebra $\mathfrak{g}$ of a Lie group $G$, then solutions remain within $G$ due to invariance under the time-ordered exponential $T\exp[\int_0^t A(\tau)d\tau]$. This is particularly important for systems with geometric or manifold constraints, such as preserving rotation or orthonormality structure in the trajectories (Tait et al., 2018).

Closed-form solutions for $x(t)$ are not generally available. Instead, the model admits only formal expansions, such as the Dyson/Neumann series, making inference intractable in closed-form.

3. Approximate Inference via Successive Approximations (Picard Iteration)

For the MLFM, inference is based on successive Picard approximations to the integral form of the dynamics: $x(t) = x(0) + \int_0^t A(\tau) x(\tau)d\tau$ A sequence $\{x_m(t)\}$ is constructed via

$x_{m+1}(t) = x(0) + \int_0^t A(\tau)x_m(\tau)d\tau$

This is discretized via quadrature at time points $\{t_i\}$: $$x_{m+1}(t_i) \approx x_m(t_0) + \sum_{j=1}^{N_i} A(\tau_{ij})\,x_m(\tau_{ij})\,w_{ij}$$ A block-operator $K[g]$ summarizes this mapping over all quadrature nodes. The update is treated as a conditional Gaussian,

$$p(x_{m+1}|x_m, g) = \mathcal{N}(x_{m+1}|K[g]x_m, \Gamma)$$

where the additive Gaussian noise $\Gamma$ reflects discretization errors. By marginalizing over intermediate approximations, one obtains a recursive prior covariance: $$\Sigma_m = K[g]\,\Sigma_{m-1}\,K[g]^{\top} + \Gamma$$ At finite $M$, this yields a tractable Gaussian approximation to $p(x_M, g)$, supporting inference by Laplace approximation, MCMC, or variational methods (Tait et al., 2018).

4. Geometric Structure and Simulation Study

The multiplicative extension is motivated by the need to enforce geometric constraints in the evolution of $x(t)$. If $\{A_r\}$ generate a Lie algebra, the flow remains within the associated group. The simulation study in (Tait et al., 2018) considers the Kubo oscillator on $\mathbb{R}^2$: $$\dot{x} = \begin{pmatrix}0 & -g(t) \ g(t) & 0\end{pmatrix} x$$ This model describes a planar rotation $x(t) = R[\int_0^t g(\tau)d\tau]\,x(0)$. The Picard successive-approximation posterior is compared to the exact (Gaussian, mod $2\pi$) posterior via the 2-Wasserstein distance. The empirical results show that increasing the Picard truncation order $M$ successively lowers the approximation error; convergence also depends on problem timescale and sampling density. For moderate $T$ and $M$, the method is practical, but longer horizons require either higher truncation order or local patching (Tait et al., 2018).

5. Relation to Classical GPLFM, Inference, and Applications

The standard additive GPLFM framework allows tractable, closed-form inference leveraging the properties of linear operations on GPs. It is routinely used for the modeling and estimation of hybrid systems where physical laws (encoded in $A,B$) are only partially known or are insufficient for capturing all observed behavior:

• Hyperparameter learning is performed by maximizing the marginal likelihood, and efficient inference is possible via Kalman filtering or GP regression.
• Interpretability is retained via the explicit physical–statistical split: mechanistic equations for known processes, GP flexibility for unknown components.
• Stochastic control and observability/controllability analyses are tractable under the standard additive GPLFM, whereas the structure for MLFMs becomes considerably more complex and problem-dependent.
• Geometric and manifold-structured dynamics (e.g., rigid body motions, rotations in SO(3), orthonormal flows) are only possible using the multiplicative latent force extension (Tait et al., 2018).

The multiplicative extension thus enables modeling richer classes of systems where the state evolution must respect group geometric constraints. However, this increased expressivity comes at the cost of lost analytic tractability, necessitating recursive or approximate methods for inference and, when available, judicious use of symmetries and group structure to simplify computation.

6. Summary and Limitations

The Gaussian Process Latent Force Model provides a parsimonious and flexible framework for integrating mechanistic differential equations with nonparametric Gaussian-process modeling of unknown inputs. The standard additive formulation admits closed-form inference and learning for linear systems, with the solution for $x(t)$ remaining Gaussian. The multiplicative latent-force extension brings control over geometric properties of the generated trajectories by embedding group structure but yields models in which direct inference is intractable; approximate methods such as truncations of Picard iteration are required, with practical convergence dependent on the chosen order and system timescale.

While the GPLFM paradigm affords direct Bayesian regression, kernel learning, and principled uncertainty quantification, its extensions to multiplicative interactions must balance computational complexity with the need for geometric fidelity. For most applications in control, identification, and learning of physical systems, the closed-form additive GPLFM remains canonical, reserving multiplicative extensions for cases where group constraints or nonlinear geometry are paramount (Tait et al., 2018).