---
title: Latent Stochastic Interpolants
url: https://www.emergentmind.com/topics/latent-stochastic-interpolants-lsi
type: topic
---

# Latent Stochastic Interpolants

Latent Stochastic Interpolants (LSI) are a theoretical and algorithmic extension of the stochastic interpolant (SI) framework, transferring SI’s continuous-time transport between probability measures from data or observation space into a learned, low-dimensional latent space. This facilitates highly flexible generative modeling, supports arbitrary latent priors, and enables efficient sampling and training schemes using a unified SDE/ODE formalism. LSI jointly optimizes the encoder, decoder, and latent-path dynamics, allowing end-to-end learning of both latent representations and the generative process bridging an arbitrary prior to the data-driven aggregated posterior. This section details the mathematical formulation, algorithmic structure, sample complexity, numerical schemes, and salient properties of the LSI framework, referencing definitions and empirical results as established across the literature, principally in [2506.02276], [2303.08797], [2508.07333], [2502.02582], and [2209.15571].

## 1. Mathematical Definition of LSI and Theoretical Foundations

Latent Stochastic Interpolants generalize SI by defining a continuous-time stochastic process in latent space via reparameterized bridges between a fixed prior $p_0(z)$ and the encoder-aggregated posterior $\int p(x_1) p_\theta(z_1|x_1) dx_1$, optimized jointly with autoencoding components. An LSI interpolant is given as
\[
z_t = \eta_t \epsilon + \kappa_t z_1 + \nu_t z_0,
\]
where $z_0 \sim p_0(z_0)$ (arbitrary prior), $z_1 \sim p_\theta(z_1|x_1)$ (learned encoder distribution), $\epsilon\sim\mathcal{N}(0, I)$, and $\{\kappa_t, \nu_t, \eta_t\}$ are scalar or vector-valued schedules parameterizing the bridge, typically satisfying
\[
\kappa_0=0,\, \kappa_1=1,\, \nu_0=1,\, \nu_1=0,\, \eta_0=\eta_1=0,\, \eta_t>0\ \text{for}\ t \in (0,1).
\]
A canonical choice is the Brownian-bridge: $\kappa_t=t,\, \nu_t=1-t,\, \eta_t=\sigma\sqrt{t(1-t)}$.

The time-marginal law of $z_t$ interpolates between $p_0$ at $t=0$ and the aggregated posterior at $t=1$ exactly in finite time. This construction enables LSI to model arbitrary transformations between latent distributions, with sampling and transport realized via an SDE or its probability-flow ODE counterpart:
\[
d z_t = h_\theta(z_t, t)\, dt + \sigma(t)\, d w_t,\qquad z_0 \sim p_0(z_0).
\]
The drift $h_\theta$ is learned to match the reference bridge determined by the interpolant.

The theoretical core is the reduction of learning a complex generative map to the problem of matching the SDE’s marginal at $t=1$ to the decoder’s latent aggregated posterior, with tightness of the continuous-time evidence lower bound (ELBO) when the variational drift equals the model drift [2506.02276, 2303.08797].

## 2. Variational Objective and Algorithmic Structure

The LSI variational objective is an ELBO derived explicitly in continuous time. Denoting $\tilde{z}_t$ as the path under the model SDE and $z_t$ as the bridge interpolation (variational posterior), the ELBO is
\[
\ln p_\theta(x_1) \geq \mathbb{E}_{q(z_t)} \left[ \ln p_\theta(x_1|z_1) - \frac{1}{2}\int_0^1 \| u(z_t, t) \|^2 dt \right],
\]
where $u$ is the optimal control,
\[
\sigma(z,t)\, u(z,t) = h_\phi(z,t) - h_\theta(z,t),
\]
with $h_\phi$ specifying the drift for the closed-form sample path defined by the bridge $z_t$.

In practice, LSI is trained in a single-stage, end-to-end manner:
- Sample mini-batch $\{x_1^{(i)}\}$, encode to stochastic $z_1^{(i)}\sim p_\theta(z_1|x_1)$, sample $z_0^{(i)} \sim p_0$ and noise $\epsilon\sim\mathcal{N}(0,I)$, and time $t \sim U[0,1]$.
- Compute $z_t$ via the interpolant.
- Evaluate ELBO terms (reconstruction and drift loss).
- Backpropagate jointly through encoder, decoder, and $h_\theta$.

A numerically robust variant is the $\mathrm{flow}$ parameterization, where the loss takes the form
\[
\mathcal{L}(\theta) = \mathbb{E}\left[-\ln p_\theta(x_1|z_1) + (\beta_t/2) \|\cdots\|^2 \right],
\]
with details in [2506.02276, eq. 19] and schedule hyperparameters (e.g. $\beta_t$) optimized per task.

## 3. Numerical Solvers and Convergence Analysis

Sampling via LSI typically integrates the learned ODE/SDE in latent space. Accurate transport is guaranteed to first- or second-order in time-step for popular integrators. The TV-distance between the true and discretized path can be upper-bounded as
\[
\text{TV} \left( \rho(t_N), \hat\rho(t_N) \right) \lesssim \cdots + \sum_{k=0}^{N-1} h_k^{2/3}[\cdots],
\]
with first-order (Euler, $O(1/N)$) and second-order (Heun/RK2, $O(1/N^2)$) error scaling, provided $\gamma(t)>0$ is handled or early-stopping avoids endpoints where $\gamma(t)\to 0$ [2508.07333].

The interpolant noise scale $\gamma(t)$ determines stiffness and grid selection; adaptive schedules where $h_k\propto\gamma(t_k)^2$ substantially improve efficiency, as can employing second-order methods with only two drift evaluations per step.

## 4. Relation to Standard Diffusion Models and Other Generative Frameworks

LSI encompasses and generalizes standard score-based diffusion and flow-matching models:
- LSI allows arbitrary prior $p_0(z_0)$ (Gaussian, uniform, Laplace, or learned), whereas diffusion models conventionally fix $p_0=\mathcal{N}(0, I)$.
- LSI runs generative dynamics in a low-dimensional latent domain, sharply reducing computational cost relative to SI or diffusion ran directly in pixel space [2506.02276 Table 1, Table 7].
- Both ODE-based (flow-matching) and SDE-based (score-based) LSI variants are supported by the same quadratic regression/ELBO objective, unifying flows and diffusions and connecting with Schrödinger bridge approaches [2303.08797, 2209.15571].
- For multimarginal or multitask settings, operator- or simplex-based generalizations of the SI time index enable one-shot, multi-way LSI frameworks, supporting inpainting, style transfer, and conditional sampling without retraining [2310.03695, 2508.04605].

## 5. Model Architectures, Parameterization, and Training Details

Empirically, in large-scale image modeling (ImageNet), the LSI pipeline comprises:
- Encoder/decoder: 2-stage convolutional stack, with up/downsampling and linear token layers.
- Latent SI (drift) network: U-Net-like stack of self-attention Transformer blocks at latent spatial resolution (e.g., $16\times 16$).
- Optimizer: AdamW $(\beta_1=0.9,~\beta_2=0.99,~\epsilon=10^{-12})$, batch size 2048, up to 2000 epochs, learning rate warmup, cosine decay, exponential moving average with 0.9999 decay.
- Time parameterization: $t\sim U[0,1]$, possibly warped as $t(s)=1-(1-s)^c$ with $c=1$.
- Noise scale $\beta$ scanned (e.g. $\beta\in[10^{-6},5\times 10^{-3}]$), with best performance for $\beta\approx 10^{-4}$ at $128^2$ resolution [2506.02276, Appendix I].

Latent interpolation schedules ($\kappa_t, \nu_t, \eta_t$) are generally chosen as linear or Brownian-bridge-like, with stochastic encoding providing regularization and improved generative diversity.

## 6. Empirical Evaluation and Benchmarks

LSI demonstrates competitive or superior performance to observation-space SI and pixel-space diffusion on standard generative metrics, using order-of-magnitude fewer sampling FLOPs. On ImageNet:
- FID for LSI ("Latent") at $128^2$ is 3.12, outperforming SI in observation space (3.46) with reduced computational budget [2506.02276, Table 1].
- Arbitrary priors (uniform, Laplace, Gaussian) deliver similar performance (e.g., FID 4.81, 4.45, 3.76) [2506.02276, Table 7].
- Classifier-free guidance and inversion via SDE/ODE sampling are effective and seamlessly supported due to the flexible latent dynamics [2506.02276, Figures 6 and 7].
- Physical emulation with LSI matches or improves upon FNO, DDPM, and FM baselines on PDE and climate tasks while requiring only $2$–$10$ integration steps for deterministic predictions or higher for calibrated ensemble spread [2509.26282].

## 7. Extensions, Limitations, and Future Directions

LSI is extensible to:
- Arbitrary prior/posterior pairs, discrete and periodic-valued spaces (e.g., with atomistic flows in crystalline materials [2502.02582]), operator-based multitask learning [2508.04605], and multimarginal translation [2310.03695].
- SDE or ODE solvers, with practical guidance to handle endpoint singularities or hyperparameter sensitivity.
- Multimodal, hierarchical, or domain-conditional latent representations [2506.02276].

Limitations noted include:
- Variational posterior $q(z_t|z_0,z_1)$ is often assumed to follow linear bridges and fixed $\sigma_t$, potentially restricting expressiveness relative to general diffusion bridges.
- Numerical issues near $t=0,1$ due to $\gamma(t)$ vanishing (endpoint singularity).
- Quantitative performance is coupled to expressive power of the autoencoder/decoder and capacity of the drift network.
- Theoretical guarantees assume correctly specified conditional expectations and sufficiently expressive function approximators.

Open research directions include learning more flexible ($t$-dependent, nonlinear) variational bridges, extension to other modalities (audio, video), tighter ELBO variants, explicit connections with Schrödinger bridge or Wasserstein gradient flows in learned latent geometries, and adaptive numerical integration strategies to further optimize sampling tradeoffs [2506.02276].

---

**References**:  
- "Latent Stochastic Interpolants" [2506.02276]  
- "Stochastic Interpolants: A Unifying Framework for Flows and Diffusions" [2303.08797]  
- "Finite-Time Convergence Analysis of ODE-based Generative Models for Stochastic Interpolants" [2508.07333]  
- "Open Materials Generation with Stochastic Interpolants" [2502.02582]  
- "Building Normalizing Flows with Stochastic Interpolants" [2209.15571]  
- "Multimarginal generative modeling with stochastic interpolants" [2310.03695]  
- "Multitask Learning with Stochastic Interpolants" [2508.04605]  
- "Reframing Generative Models for Physical Systems using Stochastic Interpolants" [2509.26282]

Source: https://www.emergentmind.com/topics/latent-stochastic-interpolants-lsi