Latent Flow Matching Models

• Latent Flow Matching Models are generative frameworks that learn continuous deterministic flows in compact latent spaces via neural ODE solvers.
• They transform noise or encoded inputs deterministically, bypassing stochastic diffusion and reducing inference steps for faster generation.
• Architectures integrate autoencoder backbones with vector field estimators, ensuring scalability, stability, and adaptability across diverse applications.

Latent flow matching models are a class of generative modeling frameworks that learn continuous deterministic flows in compact, learned latent spaces, offering notable benefits in efficiency, stability, and scalability while supporting a broad spectrum of conditional and unconditional generative tasks. The canonical latent flow matching approach formulates generation as solving an ordinary differential equation (ODE) for a low-dimensional latent representation, transporting noise or one encoding to another with a neural vector field trained via a supervised flow-matching objective. Recent advances have extended this class to diverse modalities, including images, audio, video, speech, structured data, scientific simulation, and even protein and reaction trajectory generation.

1. Core Principles of Latent Flow Matching

Latent flow matching models employ continuous deterministic flows in a learned latent space, as opposed to pixel or data-space modeling. The key mathematical formulation involves learning a vector field $v_\theta(z, t)$ on latent codes $z$, parameterized by latent time $t \in [0, 1]$, such that the ODE

$\frac{dz}{dt} = v_\theta(z, t)$

transports samples from a source latent distribution $z_0$ (often standard Gaussian or the encoded input) to a target latent $z_1$ (often an encoded sample or the canonical target distribution) (Dao et al., 2023, Liu et al., 28 Jan 2026).

The training objective is typically a supervised flow-matching loss under optimal transport: $$\mathcal{L}_{\text{flow}} = \mathbb{E}_{t \sim U[0,1]} \left[ \|v_\theta(z_t, t) - (z_1 - z_0)\|_2^2 \right]$$ where $z_t = (1-t)z_0 + t z_1$ describes the linear interpolation between the latent endpoints (Dao et al., 2023, Liu et al., 28 Jan 2026, Guan et al., 2024, Ki et al., 2024). By regressively matching the true conditional velocity, the model sidesteps the need to estimate gradients of log-densities as in score-based diffusion.

Key properties of this setup are:

• Deterministic, simulation-free ODE sampling (unlike SDE-based diffusion).
• Low-dimensional latent spaces, typically defined by a variational autoencoder (VAE) or a related deterministic encoder.
• Efficient inference and fast generation (often 10–20 ODE integration steps suffice).
• Straight-line or optimal-transport coupling between latent distributions.

2. Model Architectures and Training Workflows

Latent flow matching frameworks adopt modular architectures characterized by three main components:

1. Latent Autoencoder Backbone

• Separate or joint encoder and decoder networks, often convolutional (for images/audio) or transformer-based (for language, video, or spatiotemporal fields), are pretrained to embed data into low-dimensional latents with minimal reconstruction error (Dao et al., 2023, Askari et al., 8 Nov 2025, Ki et al., 2024, Liu et al., 28 Jan 2026).
• For conditional or domain-bridging tasks (e.g., RGB-to-RAW, image restoration), dual autoencoder branches with feature alignment losses are used for cross-domain semantic match (Liu et al., 28 Jan 2026).

2. Vector Field Estimator

• The time-dependent velocity field $v_\theta(z, t)$ is parameterized via a neural network, most often a U-Net (for images/audio), transformer (video/text), or a Fourier Neural Operator (PDE modeling), injected with time conditioning (sinusoidal or MLP-embedded), and (optionally) conditional context (Wu et al., 20 May 2025, Lee et al., 16 Oct 2025).
• Additional context (e.g., hierarchical guidance features, class labels, semantic maps, masked regions) is injected via concatenation, cross-attention, or FiLM modulation (Liu et al., 28 Jan 2026, Dao et al., 2023, Ki et al., 2024).

3. Training Regime

• Training is staged: (i) autoencoder pretraining (optionally adversarial, perceptual losses), (ii) flow-matching in latent space (with fixed or frozen autoencoder), and (iii) joint fine-tuning for end-to-end tasks (Liu et al., 28 Jan 2026).
• Loss functions combine the main flow-matching objective with reconstruction, feature alignment, perceptual, and optionally adversarial losses (Liu et al., 28 Jan 2026, Cao et al., 1 Feb 2025).

A typical RAW-to-RAW pipeline, as in RAW-Flow (Liu et al., 28 Jan 2026), combines a dual-domain latent autoencoder with cross-scale feature injection and a deterministic latent flow-matching module, yielding state-of-the-art inverse image signal processing performance.

3. Theoretical Guarantees and Convergence

Latent flow matching models have been analyzed for convergence, capacity, and expressivity under the Wasserstein-2 metric (Jiao et al., 2024, Dao et al., 2023). For a pretrained autoencoder $E$, decoder $D$, and transformer vector field, the ODE-generated distribution $\widehat{\pi}_T$ (ODE solution at time $T$) converges to the empirical data pushforward $\pi_1$ under practicable assumptions: $$\mathbb{E}\left[W_2(\widehat{\pi}_T, \pi_1)\right] \rightarrow 0 \text{ as } n \rightarrow \infty$$ where $n$ is the number of training samples, and $W_2$ is the 2-Wasserstein distance.

Approximation results demonstrate that time-dependent vector fields $v(t,z)$ can be efficiently approximated by transformers with controlled Lipschitz constants and bounded width/depth, while error rates degrade only polynomially with increased latent dimension (Jiao et al., 2024). Early stopping and Lipschitz regularization are essential for training stability and end-to-end guarantee.

4. Applications and Empirical Results

Latent flow matching models have demonstrated empirically superior or state-of-the-art performance across a wide variety of domains:

Domain	Representative Model	Latent Flow Approach	Key Empirical Highlights
Image Synth.	LFM (Dao et al., 2023)	VAE + ODE FM in latent	FID 5.26 (CelebA-HQ256@89NFE), flexible conditional schemes
Image Recon.	RAW-Flow (Liu et al., 28 Jan 2026)	Dual autoencoders + FM ODE	RAW-PSNR: 30.79dB (+2.75dB over diff/UPI baselines)
Audio Gen.	LAFMA (Guan et al., 2024)	CNF flow matching in VAE-latent	FD=31.1 (AudioCaps@N=10), ≈5x faster than diffusion
Video Gen.	VLFM (Cao et al., 1 Feb 2025), FLOAT (Ki et al., 2024)	ODE FM in latent (w/ HiPPO/poly. proj/transformer)	Interp/extrapolation at arbitrary FPS, high PSNR
Timeseries	TempO (Lee et al., 16 Oct 2025)	Latent ODE FM w/ Fourier Operator	Outperforms U-Net/ViT (MSE, spectral accuracy, efficiency)
LiDAR World	Latent CFM (Liu et al., 30 Jun 2025)	Swin-VAE latent + CFM ODE	4x–23x efficiency, SOTA IoU/mIoU, robust domain transfer
Protein Gen.	La-Proteina (Geffner et al., 13 Jul 2025)	Partially latent FM (structured)	SOTA co-designability (>800 res), functional diversity
IID	FlowIID (Singla et al., 18 Jan 2026)	VAE-guided, 1-step FM in latent	Param. efficient, real-time, SOTA on MIT/ARAP benchmarks

In each domain, latent flow matching models consistently achieve orders-of-magnitude lower inference cost (measured by number of function evaluations) and/or parameter count, while attaining or surpassing the generative fidelity and task performance of diffusion-based baselines (Schusterbauer et al., 2023, Guan et al., 2024, Cohen et al., 5 Feb 2025).

5. Extensions and Variants

Several architectural and theoretical extensions have proliferated in recent work:

• Conditional Flow Matching (CFM): Conditioning the velocity field on auxiliary data or learned latent variables extracted from the target, enabling interpretability and fine-grained control (Samaddar et al., 7 May 2025, Shen et al., 11 Feb 2026). Theoretical results guarantee that such conditioning (if implemented via feature encoding) upper-bounds the marginal CFM loss, ensuring convergence to the marginal solution (Samaddar et al., 7 May 2025).
• Stream-level and GP Stochastic Paths: Generalizing endpoint conditioning to full latent “streams” modeled by Gaussian processes (GPs), enabling low-variance marginal field estimation and tractable simulation-free training for structured, time series, and partial observation settings (Wei et al., 2024).
• Multi-domain and Cross-scale Context: Dual-domain encoding, cross-scale context guidance, and feature fusion constrain flow matching to respect domain alignment (e.g., RGB/RAW pairs (Liu et al., 28 Jan 2026)), improving transferability and feature restoration.
• Partially Latent Flows: For complex domains (e.g., protein structure), splitting explicit coordinates (e.g., $C_\alpha$ backbone) and high-capacity per-entity latents (e.g., side-chains/sequence) enables scalable, structured conditional flow matching (Geffner et al., 13 Jul 2025).
• Efficiency Variants: Multi-segment and consistency-enforcing objectives (as in ELIR (Cohen et al., 5 Feb 2025)) further accelerate inference and stabilize training, while deterministic ODE integrators make on-device deployment practical.

6. Practical Implications, Limitations, and Future Directions

Latent flow matching models realize significant improvements in computational efficiency and scalability, attributed to operating in low-dimensional learned manifolds, leveraging ODE-solver–based deterministic transport, and sidestepping stochasticity and iterative denoising inherent in SDE-based diffusion (Dao et al., 2023, Schusterbauer et al., 2023). These models are especially well-suited for high-resolution or resource-constrained scenarios (edge/real-time), domain adaptation, and multi-task transfer (Liu et al., 30 Jun 2025, Singla et al., 18 Jan 2026).

Key practical guidelines, as supported by theory and empirical ablation:

• Lower latent dimensionality (sufficient for the signal) accelerates convergence and increases statistical efficiency.
• Transformer-based or U-Net-based vector fields (with controlled Lipschitz constants) yield stable latent ODEs and universal approximation properties (Jiao et al., 2024).
• Cross-scale, context, and condition injection are critical for handling ill-posed inverse problems and cross-domain transfer (Liu et al., 28 Jan 2026).
• Straight-line optimal-transport interpolation plus flow-matching regression are sufficient for highly effective generative transport—complex SDE or diffusion-based perturbations are not required in compact latent representations.

Limitations persist in settings with extremely lossy autoencoding or when latent compression discards essential signal, as the flow matching cannot reconstruct what is irretrievably lost. Addressing non-Lambertian/complex reflectances, generalization outside the pre-trained latent manifold, and memory bottlenecks for extremely long sequences or high-resolution video remain open research directions (Singla et al., 18 Jan 2026, Cao et al., 1 Feb 2025).

Extensions anticipated include:

• Joint end-to-end training of autoencoder, latent flow, and contextual modules.
• Hierarchical or attention-based context integration for structured or multi-modal data.
• Learned or adaptive integration schedules for improved numerical and sample efficiency.
• Generalization to new tasks such as missing data imputation, structure-conditioned design, or large-language-model compression (Wu et al., 20 May 2025).

7. Representative Models and Summary Table

The following table summarizes several representative latent flow matching models, their latent type, primary architecture, and characteristic empirical performance:

Model	Latent Type	Velocity Network	Application/Task	Key Metric/Result
LFM (Dao et al., 2023)	VAE-latent	DiT, ADM-UNet	Uncond./Cond. Image Gen.	FID 5.26 (CelebA-HQ 256)
RAW-Flow (Liu et al., 28 Jan 2026)	Dual-branch latent	UNet, DLAE	RGB→RAW inv. ISP	+2.75dB PSNR over SOTA
LAFMA (Guan et al., 2024)	Conv VAE-mel	UNet	Text-to-Audio	FD=31.1, 10 ODE steps
VLFM (Cao et al., 1 Feb 2025)	Patch-based latent	DiT (HiPPO)	Text-to-Video	High PSNR, robust interp/extrap
TempO (Lee et al., 16 Oct 2025)	PCA/autoenc latent	FNO, Unet	PDE Timeseries Forecast	SOTA spectrum/PSNR
ELIR (Cohen et al., 5 Feb 2025)	TinyAE	Conv UNet	Image Restoration	FID [email protected], ~4x smaller
La-Proteina (Geffner et al., 13 Jul 2025)	Hybrid (exp+latent)	Pair-biased Transformer	All-atom Protein Gen.	SOTA co-design, >800aa
FlowIID (Singla et al., 18 Jan 2026)	VAE-guided shading	UNet (single-step)	Intrinsic Image Decomp.	SOTA param./runtime efficiency
LatentRxnFlow (Shen et al., 11 Feb 2026)	GNN latent	MLP, FiLM	Reaction Trajectory Modeling	SOTA, interpretable trajectories

Empirical best practices and theoretical guarantees favor latent flow matching as an efficient and reliable paradigm for high-dimensional generative modeling when suitable latent representations are available or can be learned.

---

References:

• (Dao et al., 2023) Flow Matching in Latent Space
• (Jiao et al., 2024) Convergence Analysis of Flow Matching in Latent Space with Transformers
• (Schusterbauer et al., 2023) Boosting Latent Diffusion with Flow Matching
• (Ki et al., 2024) FLOAT: Generative Motion Latent Flow Matching for Audio-driven Talking Portrait
• (Cohen et al., 5 Feb 2025) Efficient Image Restoration via Latent Consistency Flow Matching
• (Liu et al., 28 Jan 2026) RAW-Flow: Advancing RGB-to-RAW Image Reconstruction with Deterministic Latent Flow Matching
• (Guan et al., 2024) LAFMA: A Latent Flow Matching Model for Text-to-Audio Generation
• (Liu et al., 30 Jun 2025) Towards foundational LiDAR world models with efficient latent flow matching
• (Askari et al., 8 Nov 2025) Latent Refinement via Flow Matching for Training-free Linear Inverse Problem Solving
• (Geffner et al., 13 Jul 2025) La-Proteina: Atomistic Protein Generation via Partially Latent Flow Matching
• (Samaddar et al., 7 May 2025) Efficient Flow Matching using Latent Variables
• (Shen et al., 11 Feb 2026) Driving Reaction Trajectories via Latent Flow Matching
• (Wu et al., 20 May 2025) Latent Flow Transformer