---
title: Multiplicative Latent Force Model (MLFM)
url: https://www.emergentmind.com/topics/multiplicative-latent-force-model-mlfm
type: topic
---

# Multiplicative Latent Force Model (MLFM)

The Multiplicative Latent Force Model (MLFM) is an extension of the latent force model framework designed to embed intricate geometric constraints and manifold-valued dynamics within hybrid state-space models. MLFM couples mechanistic ordinary differential equations (ODEs) with multiplicative interactions between the state and latent Gaussian process (GP) dynamics, enabling the principled modeling of curved or group-structured trajectories—but introducing significant inference challenges absent in classical, additive LFM settings [1812.11755][1811.00423].

## 1. Model Definition and Formalism

The MLFM describes a $K$-dimensional latent state $x(t) \in \mathbb{R}^K$ evolving according to the non-autonomous linear ODE
\[
\frac{d x(t)}{dt} = A(t)\,x(t),
\]
where the time-varying coefficient matrix $A(t)$ is itself a linear function of latent Gaussian processes:
\[
A(t) = A_0 + \sum_{r=1}^R A_r\,g_r(t),
\]
with each $A_r \in \mathbb{R}^{K \times K}$. The latent forces $g_r(t)$ are independent GPs, typically with squared-exponential (RBF) kernels
\[
g_r(t) \sim \mathcal{GP}\bigl(0,\,k_r(t,t';\phi_r)\bigr), \quad\text{where}\quad k_r(t, t') = \sigma_r^2 \exp\left(-\frac{(t-t')^2}{2\ell_r^2}\right).
\]
This model generalizes the additive LFM, where $A(t)$ would be replaced by $D$ (diagonal) and an additive $S g(t)$ term, resulting in solutions where $x(t)$ remains jointly Gaussian with the GPs. In contrast, the multiplicative form forces the dependency between $x$ and $g$ to be nonlinear due to the interaction of $A(t)$ and $x(t)$ [1812.11755][1811.00423].

Often, the structure matrices $A_r$ are further expanded as
\[
A_r = \sum_{d=1}^D \beta_{rd}\,L_d, \quad L_d \in \mathbb{R}^{K \times K},
\]
where the “connection” matrix $\beta$ combines a fixed matrix basis $\{L_d\}$, often drawn from a Lie algebra $\mathfrak{g}$. This guarantees that $A(t) \in \mathfrak{g}$ for all $t$, so the solution map (the fundamental solution) $\Phi(t)$ lies in the associated Lie group $G$, constraining the trajectories to a specific manifold or group orbit [1812.11755].

## 2. Geometric Implications and Trajectory Structure

The intrinsic geometry of MLFM trajectories is determined by the algebraic structure of the $A_r$ matrices. When these belong to a Lie algebra $\mathfrak{g}$ (e.g., $\mathfrak{so}(3)$ for rotation), the solution
\[
\Phi(t)=\mathcal{T} \exp\left(\int_0^t A(\tau) d\tau\right),
\]
ensures $x(t)=\Phi(t)x(0)$ evolves within the Lie group $G$. This property builds geometric constraints such as norm preservation or group actions directly into the model. The trajectories do not generally constitute a jointly Gaussian process over $\mathbb{R}^K$, and their structure can encode rotations (Kubo oscillators), quaternions ($S^3$ for orientation), and other non-Euclidean evolutions. By contrast, in standard LFM, $x(t)$ describes a linear-Gaussian process in Euclidean space [1811.00423][1812.11755].

## 3. Inference: Successive Approximations and Mixtures

Inference in MLFMs is complex due to the loss of closed-form solutions for the marginal likelihood of $x$ given observations and the nonlinear dependence on the latent GPs. Multiple tractable approximate inference strategies have been proposed:

### a. Successive Approximations (Picard Iteration)

The solution of the ODE can be represented as the Picard integral:
\[
x(t) = x(0) + \int_0^t A(\tau) x(\tau) d\tau.
\]
Discretizing this integral and applying $M$ iterations yields a block-matrix mapping $K[g]$, which is then used to construct the $M$th-order approximation to the trajectory and its covariance via recursion:
\[
\Sigma_m = K[g]\, \Sigma_{m-1}\, K[g]^\top + \Gamma, \qquad \Sigma_0 \text{ given}.
\]
Likelihood evaluation and optimization over $g$ (and optionally hyperparameters) proceed by recursively propagating this covariance and forming Laplace-style approximations for the joint posterior [1811.00423].

### b. Mixtures of Successive Approximations (MLFM–MixSA)

To improve local accuracy for long or non-uniformly spaced trajectories, expansions are mixed over multiple anchor points (“local centers”). At each anchor $\tau_\nu$, a local Picard expansion of mean $m_\nu$ is built; the conditional density is then
\[
p_\nu(X \mid g, \beta, \alpha) = \prod_{i=1}^N \mathcal{N}\bigl(x(t_i) \mid m_{\nu, i}, \alpha^{-1} I_K\bigr),
\]
and the full likelihood is a mixture
\[
p(X \mid g, \beta, \alpha) = \sum_{\nu=1}^D \pi_\nu\, p_\nu(X \mid g, \beta, \alpha).
\]
An EM algorithm alternates between estimating responsibilities and maximizing the likelihood [1812.11755].

## 4. Adaptive Gradient Matching

As an alternative, the adaptive gradient matching strategy (MLFM–AG) operates by enforcing consistency between an interpolating GP prior placed on the state path $x(t)$ and the ODE-implied derivatives:
- Each coordinate $x_k$ receives a GP prior, resulting in a conditional Gaussian law for the derivative vector $\dot{x}_k$.
- A “regression expert” likelihood for the ODE model error is specified as a Gaussian about the modelled derivative $f_k$.
- The product of the GP and regression expert yields a joint Gaussian approximating the likelihood of the observed path.
- In the linear MLFM case (where $A(t)$ is affine in $g$ with fixed $L_d$ and $\beta$), the full conditionals for $X$, $g$, and $\beta$ given the others are Gaussian, permitting efficient Gibbs or mean-field updates. For each $g$, the joint posterior is explicitly computable; the same applies for $\beta$ [1812.11755].

## 5. Empirical Evaluation and Benchmarking

Quantitative assessment of MLFM inference approaches has been conducted using both simulated and real-world data:

- **Kubo Oscillator** ($z \in S^1$): Simulation studies compare MLFM–AG and MLFM–MixSA by $\ell_2$-distance between the true $g$ and its MAP estimate. MLFM–AG achieves high accuracy at small sampling intervals, but deteriorates as data become sparse. MLFM–MixSA retains accuracy for sparse sampling, particularly with increased mixture centers and expansion order [1812.11755], [1811.00423].
  
- **Dynamics on $\mathrm{SO}(3)$**: MLFM–MixSA achieves lower reconstruction errors than MLFM–AG on sparsely observed rotational data, with improvements saturating at expansion orders $M \geq 5$.
  
- **Human Motion-Capture (MOCAP) Data**: MLFM is applied to the modeling of joint trajectories represented as quaternions ($S^3$). Marginal MLFMs with $R=2$ latent forces suffice for individual joints. Coupled ("product MLFM") models with shared latent GPs ($R=4$) yield superior predictive accuracy, reflecting the pooling of information across coupled manifold-valued paths.

Benchmarking results for the Wasserstein-2 distance between the true and approximate GP posterior for the Kubo oscillator, as a function of truncation order $M$, interval $T$, and sample step $\Delta t$, show a monotonic drop in error with increasing $M$, and better performance for shorter time horizons [1811.00423].

## 6. Computational Properties and Practical Considerations

A comparison of inference schemes is summarized below:

| Method         | Accuracy       | Computational Complexity                  | Scalability & Suitability         |
|----------------|---------------|-------------------------------------------|-----------------------------------|
| MLFM–AG        | High (dense data) | $O(N^3)$ for GP plus low-dimensional updates | Good for large $K$, $R$, well-sampled data   |
| MLFM–MixSA     | High (sparse data) | $O(M\,(NK)^3)$, larger for more anchors, $M$ | Challenging for high $K$ or long series |

MLFM–AG is fast and effective when observation grids are dense enough that GP interpolation is accurate, but loses geometric fidelity when the state manifold is strongly curved (e.g., spheres, rotations) or sampling is sparse. MLFM–MixSA, using localized Taylor–Picard expansion mixtures, remains robust in these challenging regimes at the price of increased computation [1812.11755]. In practice, MLFM–AG is recommended as a default, with MLFM–MixSA employed in high-curvature or sparsely sampled scenarios.

## 7. Summary of Implementation Approach

Implementation proceeds via:
1. Discretization of observation times $\{t_i\}$, quadrature nodes and weights.
2. Initialization of prior distributions for $x_0$, GP on $g$.
3. Construction of block-matrix operators $K[g]$ for each $g$.
4. Recursion of covariance propagation for $M$ steps, yielding marginal likelihood via sequenced Gaussian updates or direct optimization.
5. Parameter and hyperparameter updating by (gradient) optimization or EM, as dictated by chosen approximation [1811.00423].

These procedure details are summarized in implementation pseudocode in [1811.00423], supporting Laplace-style MAP optimization.

---

The MLFM provides a flexible, geometrically expressive class of latent dynamical models for manifold-valued systems, at the cost of nontrivial but tractable approximate inference. Empirical results confirm suitability for rotational dynamics, coupled manifold states, and scenarios requiring the integration of differential geometry with modern machine learning techniques [1812.11755][1811.00423].

Source: https://www.emergentmind.com/topics/multiplicative-latent-force-model-mlfm