Latent Conditional Flow Matching (LCFM)
- LCFM is a latent-space approach to flow matching that trains a time-dependent velocity field to transport latent codes conditioned on auxiliary data.
- It is applied across diverse modalities, including image synthesis, medical segmentation, audio super-resolution, and reaction prediction for efficient generation.
- The framework uses staged training and ODE-based sampling with task-specific latent representations to achieve state-of-the-art performance.
Latent Conditional Flow Matching (LCFM) denotes conditional flow matching executed in a learned latent space rather than directly in the observation space. In the formulations collected across recent work, a time-dependent velocity field is learned to transport a simple source distribution or a task-specific source latent toward a target latent distribution, while conditioning on auxiliary information such as images, reactants, local maps, low-resolution audio, score tokens, or multimodal embeddings. Conceptually, LCFM is the latent-space instantiation of Flow Matching (FM) and Conditional Flow Matching (CFM): FM trains a continuous-time generative model by regressing a vector field to the velocity of a prescribed probability path, and CFM extends that construction to conditional distributions (Lipman et al., 2022). Latent-space variants were made explicit for pretrained autoencoders and conditional generation in image synthesis (Dao et al., 2023), for latent-variable conditioning with pretrained VAEs or GMMs (Samaddar et al., 7 May 2025), and for domain-specific conditional transport problems such as medical image segmentation, where the method is named “LatentFM” even though the paper does not explicitly use the acronym “LCFM” (Ngoc et al., 4 Dec 2025).
1. Historical emergence and conceptual scope
FM was introduced as a simulation-free approach for training Continuous Normalizing Flows by regressing vector fields of fixed conditional probability paths, rather than by simulating the ODE during training (Lipman et al., 2022). The same framework established that FM is compatible with a general family of Gaussian probability paths and that it also supports non-diffusion paths, including Optimal Transport displacement interpolation (Lipman et al., 2022). That construction is the immediate precursor of LCFM.
The shift from pixel-space or waveform-space FM to latent-space FM was motivated by computational efficiency, scalability, and the observation that many real datasets reside on lower-dimensional manifolds. “Flow Matching in Latent Space” proposed applying flow matching in the latent spaces of pretrained autoencoders and explicitly integrated conditions for label-conditioned image generation, image inpainting, and semantic-to-image generation (Dao et al., 2023). “Efficient Flow Matching using Latent Variables” then formalized a latent-variable-conditioned variant, denoted Latent-CFM, in which a pretrained latent variable model supplies a latent feature that conditions the velocity field and induces a mixture decomposition of the endpoint coupling (Samaddar et al., 7 May 2025).
By 2025–2026, equivalent or closely related latent conditional flow constructions appeared in multiple application domains. These include medical image segmentation with separate VAEs for images and masks (Ngoc et al., 4 Dec 2025), reaction prediction with graph latent trajectories anchored at the thermodynamic product state (Shen et al., 11 Feb 2026), local navigation beyond field of view through latent map transport (Park et al., 3 Mar 2026), audio super-resolution in continuous audio latent space (Liu et al., 10 Apr 2026), neural codec latent bandwidth extension (Zhang et al., 2 Mar 2026), expressive singing voice synthesis through prior-to-posterior latent transport (Yun et al., 1 Jan 2026), and multimodal music generation in a pretrained Mel-spectrogram VAE latent space (Song et al., 18 Apr 2025). This distribution of use suggests that LCFM is better understood as a modeling pattern than as a single architecture.
2. Mathematical formulation
The common starting point is the flow ODE
or, in latent space,
with . In FM, the learned vector field is trained to match the target velocity associated with a prescribed probability path, and the induced marginals satisfy the continuity equation
(Lipman et al., 2022). CFM introduces conditioning on an auxiliary variable , yielding or , depending on notation.
A canonical bridge used repeatedly in LCFM is the straight-line interpolation between a source endpoint and a target endpoint. In the simplest two-endpoint latent construction,
and the regression objective takes the form
This exact pattern appears in latent medical segmentation, where 0 is sampled from a simple prior and 1 is the latent mask code conditioned on the latent image code 2 (Ngoc et al., 4 Dec 2025). A related conditional formulation appears in latent-variable image generation: 3 with an additional KL regularizer on the latent encoder in the VAE-based version (Samaddar et al., 7 May 2025).
Not all LCFM instantiations use the same bridge. “LatentRxnFlow” uses a noisy linear bridge in graph latent space,
4
with constant target velocity 5 (Shen et al., 11 Feb 2026). “LatentFlowSR” uses Optimal Transport Conditional Flow Matching with a deterministic zero-variance path,
6
and trains
7
(Liu et al., 10 Apr 2026). The coexistence of linear deterministic bridges, Gaussian bridges, and “thickened” noisy bridges indicates that LCFM is defined by latent-space conditional velocity matching rather than by a unique path family.
3. Latent representations and conditioning mechanisms
LCFM depends on a latent representation that makes conditional transport tractable. In the image and medical segmentation literature, this typically means a pretrained or separately trained autoencoder or VAE. “Flow Matching in Latent Space” uses the pretrained Stable Diffusion VAE, with latents 8 and a decoder 9 back to image space (Dao et al., 2023). “LatentFM” uses two VAEs: one for medical images and one for segmentation masks, with priors 0 and 1, and with reparameterized latent codes 2 and 3 (Ngoc et al., 4 Dec 2025). “LatentFlowSR” uses a noise-robust autoencoder that maps high-resolution and low-resolution audio to continuous latents 4, with 5 channels (Liu et al., 10 Apr 2026).
Other domains replace VAEs with task-specific latent encoders. “LatentRxnFlow” uses a graph autoencoder-like encoder/decoder sandwich: an encoder maps reactant and product molecular graphs to node-wise latent representations, and a decoder reconstructs product bonds and atom properties from latent states (Shen et al., 11 Feb 2026). “DreamFlow” encodes local heightmaps and broader spatial maps into latent vectors 6 and 7 for navigation (Park et al., 3 Mar 2026). “CodecFlow” uses continuous neural codec embeddings 8 with 9, followed by a structure-constrained residual vector quantizer before waveform decoding (Zhang et al., 2 Mar 2026). “FM-Singer” operates between a score-conditioned prior 0 and a recording-conditioned posterior 1 in a cVAE backbone, with latent channel dimensionality 192 per frame (Yun et al., 1 Jan 2026).
Conditioning mechanisms vary with modality. In medical segmentation, 2 and 3 are fed jointly to a UNet velocity network together with a time embedding of 4 (Ngoc et al., 4 Dec 2025). In reaction prediction, the vector field 5 is modulated by FiLM, with reaction conditions encoded as a concatenation of a Top-K frequent-agent multi-hot vector and a DeepSets embedding of ECFP4 fingerprints (Shen et al., 11 Feb 2026). In multimodal music generation, image, story, and caption embeddings are aligned into the audio CLAP space via three MLP adapters and averaged into a fused embedding 6, which is then supplied to a Transformer-UNet latent velocity model 7 (Song et al., 18 Apr 2025). In neural codec bandwidth extension, the condition is 8, where 9 is a frame-level voicing sequence, fused through a convolutional projection into a voicing-aware condition 0 (Zhang et al., 2 Mar 2026). In label-conditioned latent image generation, classifier-free guidance is implemented directly on the velocity field,
1
4. Training, sampling, and diagnostic use
A recurrent training pattern is staged optimization: first train or freeze the latent representation model, then train the latent flow. “LatentFM” explicitly uses two-stage training: first train the two VAEs, then fix them and train the LCFM model on latent codes (Ngoc et al., 4 Dec 2025). “LatentFlowSR” first pretrains the noise-robust autoencoder and freezes it before training the CFM velocity field for 2M steps with AdamW (Liu et al., 10 Apr 2026). “CodecFlow” uses three-stage training: codec pretraining, latent CFM training for the Flow Embedding Converter, and end-to-end fine-tuning of the codec encoder/decoder while keeping the converter and SC-RVQ fixed (Zhang et al., 2 Mar 2026). “FM-Singer” adds the latent CFM regression loss to a cVAE-GAN training objective that already includes adversarial, feature-matching, mel, KL, DSP, duration, and auxiliary prior-side losses (Yun et al., 1 Jan 2026).
Sampling is always ODE-based, but the initial condition depends on the task. In latent generative modeling from a simple prior, sampling begins from 3 and integrates to 4 before decoding (Liu et al., 10 Apr 2026). In latent image synthesis with a pretrained autoencoder, sampling inverts the latent flow from 5 to 6 and then decodes 7 (Dao et al., 2023). In task-conditioned transport problems, the source state is often itself meaningful: in reaction prediction, the initial state is the encoded reactant graph 8 and the terminal state approximates the encoded product 9 (Shen et al., 11 Feb 2026); in DreamFlow, integration starts from the local observation latent 0 and predicts the broader latent 1 (Park et al., 3 Mar 2026).
Several works use LCFM outputs not only for generation but also for uncertainty or trajectory analysis. In medical segmentation, 2 latent mask samples are decoded, the pixel-wise mean
3
is thresholded at 4 for the final mask, and the pixel-wise variance
5
is reported as a confidence or uncertainty map (Ngoc et al., 4 Dec 2025). In “LatentRxnFlow,” the full latent trajectory is analyzed via speed, arc length, path inefficiency, mean curvature, minimum alignment, latent kinetic energy, terminal speed, and structural dwell time; those geometric descriptors are then used for failure localization and gated inference (Shen et al., 11 Feb 2026). DreamFlow reports in-distribution and out-of-distribution cosine similarity between predicted and ground-truth latents and couples the predicted broader latent context to a DRL local navigation policy (Park et al., 3 Mar 2026). A plausible implication is that, within LCFM, the latent trajectory itself often becomes an object of analysis rather than merely a hidden computation.
5. Domain-specific instantiations and empirical record
In generative medical image segmentation, “LatentFM” was evaluated on ISIC-2018 and CVC-ClinicDB. On the test sets, the reported Dice/IoU of LatentFM were 6 on ISIC and 7 on CVC, compared with pixel-space FM at 8 and 9, respectively. The paper also reports mask-VAE reconstructions with Dice/IoU 0, SSIM 1, and PSNR 2, and uses 3 samples per image “to mimic five clinician opinions, following MedSegDiff” (Ngoc et al., 4 Dec 2025).
In reaction prediction, “LatentRxnFlow” reports on USPTO-MIT that the full FiLM-conditioned model reaches Top-1 4, Top-2 5, Top-3 6, Top-5 7, and Top-10 8. On RTX3090, the reported latency is 9 ms with RK4 and 0 steps, and 1 ms with 2 steps, while the paper states that the cost of increasing ODE steps is negligible because inference is dominated by single encode/decode passes (Shen et al., 11 Feb 2026).
In local navigation, DreamFlow compares latent predictors by cosine similarity. The reported in-distribution and out-of-distribution values are 3 and 4 for an MLP, 5 and 6 for FM without conditioning, and 7 and 8 for FM with conditioning, with inference times 9, 0, and 1, respectively (Park et al., 3 Mar 2026).
In audio super-resolution, “LatentFlowSR” uses a one-step ODE solver in latent space and reports that the full model requires 2 inference step, 3 M parameters, and 4 G FLOPs per 5 s audio, compared with AudioSR at 6 steps, 7 M parameters, and 8 G FLOPs, FlashSR at 9 step and 0 G FLOPs, and FlowHigh at 1 step and 2 G FLOPs. Subjective MOS at 3 kHz are reported as 4 on VCTK, 5 on ESC-50, 6 on internal music, and 7 on MUSDB18-HQ (Liu et al., 10 Apr 2026).
In speech bandwidth extension, “CodecFlow” reports for 8 kHz: LSD 9, LSD-LF 00, LSD-HF 01, ViSQOL 02, MOS 03, and COL 04. For 05 kHz, the reported values are LSD 06, LSD-LF 07, LSD-HF 08, ViSQOL 09, MOS 10, and COL 11 (Zhang et al., 2 Mar 2026).
In expressive singing voice synthesis, “FM-Singer” reports on a Korean dataset after 12k steps: VISinger2 with MCD 13, F0 RMSE 14, MOS 15; VISinger2 NF with MCD 16, F0 RMSE 17, MOS 18; and FM-Singer with MCD 19, F0 RMSE 20, MOS 21. On OpenCpop after 22k steps, FM-Singer reports MCD 23 and F0 RMSE 24 (Yun et al., 1 Jan 2026).
In multimodal music generation, MusFlow is trained in the pretrained MusicLDM VAE latent space and reports, for caption-to-music, FAD 25, KL 26, and CLAP 27; for story-to-music, FAD 28, KL 29, and CLAP 30; for image-to-music, FAD 31, KL 32, and ImageBind 33; and for multimodal generation, FAD 34, CLAP 35, and ImageBind 36. Subjective scores are reported as OVL 37 and REL 38 (Song et al., 18 Apr 2025).
6. Theory, comparative positioning, and limitations
The most general theoretical extension appears in “Latent Process Generator Matching,” which treats the observed generative state as a deterministic image 39 of a tractable Markov process 40. The paper proves that the projected generator
41
generates the one-time marginals of the projected process, and it gives gradient equality between conditional latent-process losses and the corresponding marginal loss on 42 (Billera et al., 19 May 2026). In that framework, LCFM is the deterministic-flow special case in which projected drifts or velocities are learned in the observed space while conditioning on latent trajectories only during training (Billera et al., 19 May 2026).
A second theoretical result appears in “Flow Matching in Latent Space,” which proves
43
thereby upper-bounding the Wasserstein-2 distance between the generated distribution and the true data distribution by reconstruction error and the latent FM regression error (Dao et al., 2023). “Efficient Flow Matching using Latent Variables” proves an upper-bound relation between the latent-conditioned objective and the CFM objective: 44 which the paper uses to justify minimizing LCFM as a surrogate for CFM under latent-variable conditioning (Samaddar et al., 7 May 2025).
Across the surveyed papers, LCFM is positioned against diffusion, GANs, and standard normalizing flows in a fairly consistent way. FM is described as learning exact densities via regression to a path-induced velocity while avoiding score matching and stochastic simulation (Ngoc et al., 4 Dec 2025). It is also described as avoiding adversarial training instability and mode collapse, and as not being constrained by invertible architectures or tractable Jacobians in the way normalizing flows are (Ngoc et al., 4 Dec 2025). Reaction, robotics, and audio papers emphasize deterministic ODE transport, analyzable trajectories, and reduced sampling cost relative to diffusion-style reverse-time procedures (Shen et al., 11 Feb 2026, Park et al., 3 Mar 2026, Liu et al., 10 Apr 2026). This suggests a broad methodological pattern: latent-space FM is being used where simulation-free regression and lower-dimensional integration are operationally preferable to high-dimensional stochastic generation.
The limitations reported in the literature are likewise recurrent. Multiple papers state that performance depends on the quality of the latent representation or decoder; poor reconstructions can bottleneck downstream conditional generation (Ngoc et al., 4 Dec 2025, Liu et al., 10 Apr 2026). Straight-line or constant-velocity paths may be suboptimal for complex manifolds: “LatentFM” notes that the linear path 45 may be suboptimal for very complex mask manifolds (Ngoc et al., 4 Dec 2025), while “LatentRxnFlow” notes that 46 may bias to straight latent routes and lacks a sink at 47, so overshooting can occur (Shen et al., 11 Feb 2026). Robotics and audio papers note sensitivity to domain shift and out-of-distribution conditions (Park et al., 3 Mar 2026, Liu et al., 10 Apr 2026). Uncertainty quantification is often informative but heuristic: medical segmentation uses sample variance maps, and reaction prediction uses geometric trajectory proxies whose correlations are empirical rather than calibrated probabilities (Ngoc et al., 4 Dec 2025, Shen et al., 11 Feb 2026).
Within these bounds, LCFM has become a general recipe for conditional transport in learned latent spaces: choose a latent representation, specify a conditional path, regress a time-dependent velocity field, solve an ODE at inference, and decode or otherwise consume the transported latent. The precise latent variable, the conditioning interface, and the path family vary by task, but the core construction remains stable across the current literature (Lipman et al., 2022).