Flow-Matching MCMC: Adaptive Global Sampling

• The paper presents FM-MCMC as a hybrid sampler that integrates a flow-matching-trained continuous normalizing flow with traditional MCMC to supply non-local proposals while maintaining target invariance.
• It alternates between gradient-based local moves and global, flow-informed transitions, using adaptive training from chain-generated samples to enhance exploration in multimodal, high-dimensional problems.
• Adaptive tempering and Metropolis corrections are key to balancing computational efficiency and exactness, despite challenges from nonconvex flow-learning objectives.

Searching arXiv for papers on Flow-Matching MCMC and closely related methods. Flow-Matching Markov Chain Monte Carlo (FM-MCMC) denotes a class of hybrid samplers that embed a flow-matching-trained continuous normalizing flow within a Markov chain Monte Carlo procedure, so that a learned transport supplies non-local proposals while MCMC preserves exact target invariance. In the formulation introduced in "Markovian Flow Matching: Accelerating MCMC with Continuous Normalizing Flows" (Cabezas et al., 2024), the target is an unnormalized density $\pi(x)=\hat\pi(x)/Z$, the flow is adapted on-the-fly from samples produced by the chain, and the resulting sampler combines a local Markov transition kernel with a non-local, flow-informed transition kernel. The method is motivated by the inefficiency of conventional local samplers on high-dimensional, rugged, and especially multimodal targets, and by the possibility of using flow matching as a comparatively inexpensive way to learn global transport structure (Cabezas et al., 2024).

1. Formal setting and defining construction

In FM-MCMC, a time-dependent vector field $v_t$ defines a continuous normalizing flow $\phi_t$ through the ODE

$$\frac{\mathrm{d}}{\mathrm{d}t}\phi_t(x)=v_t(\phi_t(x)),\qquad \phi_0(x)=x,$$

with induced probability path $p_t=[\phi_t]_\sharp p_0$. Along this path, the instantaneous change-of-variables formula is

$$\log p_t(x_t)=\log p_0(x)-\int_0^t \nabla\cdot v_s(x_s)\,\mathrm{d}s, \qquad x_s:=\phi_s(x).$$

The distinctive move made by FM-MCMC is not to use the CNF as a standalone approximate generator, but to use it inside an MCMC kernel as a learned proposal mechanism. In the primary formulation, the sampler alternates a local kernel $Q$ with a non-local flow-informed kernel $P$, and the combined transition is written as

$R:=P\circ Q,$

or explicitly,

$$R(x,\mathrm{d}y;\pi,\theta)=\int Q(x,\mathrm{d}z;\pi)\,P(z,\mathrm{d}y;\pi,\theta).$$

Operationally, the algorithm performs $v_t$0 local steps with $v_t$1 and then one flow-informed step with $v_t$2 (Cabezas et al., 2024).

This construction places FM-MCMC between two established paradigms. It is not standard MCMC, because the proposal mechanism is learned rather than fixed or hand-designed. It is also not a pure flow sampler, because every global proposal is Metropolized. A plausible implication is that FM-MCMC should be understood less as a replacement for MCMC than as an adaptive transport-enhanced MCMC architecture.

2. Flow matching objective and conditional probability paths

The learning component is based on flow matching. The classical objective is

$v_t$3

but because the target vector field is unavailable, the method uses the conditional flow matching objective

$v_t$4

For the conditional path, the paper uses the optimal-transport path from Lipman et al., with

$v_t$5

so that

$v_t$6

and

$v_t$7

Equivalently,

$v_t$8

This yields the Monte Carlo estimator

$v_t$9

where $\phi_t$0 and $\phi_t$1 (Cabezas et al., 2024).

The defining feature here is that the conditional targets $\phi_t$2 are not assumed to be available as i.i.d. samples beforehand. Instead, the chain itself produces the samples used to train the flow. This gives FM-MCMC its adaptive, self-referential character: the MCMC component supplies pseudo-target samples, and those samples in turn refine the proposal flow (Cabezas et al., 2024).

3. Hybrid transition kernels and exactness through Metropolization

The local transition kernel $\phi_t$3 is a standard gradient-based MCMC kernel. In the reported experiments it is MALA:

$\phi_t$4

with proposal density

$\phi_t$5

and acceptance probability

$\phi_t$6

The non-local kernel $\phi_t$7 is a flow-informed random-walk Metropolis-Hastings kernel:

$\phi_t$8

where $\phi_t$9 is induced by mapping the current state backward through the learned flow to reference space, perturbing there with a Gaussian random walk, and mapping forward again. Concretely,

$$\frac{\mathrm{d}}{\mathrm{d}t}\phi_t(x)=v_t(\phi_t(x)),\qquad \phi_0(x)=x,$$0

The Metropolis-Hastings acceptance probability is

$$\frac{\mathrm{d}}{\mathrm{d}t}\phi_t(x)=v_t(\phi_t(x)),\qquad \phi_0(x)=x,$$1

The appendix also gives the optimal proposal scale

$$\frac{\mathrm{d}}{\mathrm{d}t}\phi_t(x)=v_t(\phi_t(x)),\qquad \phi_0(x)=x,$$2

A corresponding pullback relation is

$$\frac{\mathrm{d}}{\mathrm{d}t}\phi_t(x)=v_t(\phi_t(x)),\qquad \phi_0(x)=x,$$3

These formulas show how the learned CNF supplies non-local proposals without sacrificing exact invariance of the target (Cabezas et al., 2024).

This hybridization is the central reason FM-MCMC differs from direct flow-based generation. The flow is not trusted unconditionally; it is embedded in a correction mechanism. That design choice also situates FM-MCMC alongside earlier flow-plus-MCMC hybrids, including "MetFlow: A New Efficient Method for Bridging the Gap between Markov Chain Monte Carlo and Variational Inference" (Thin et al., 2020), where proposals are obtained using normalizing flows inside a Metropolis-Hastings framework.

4. Adaptive training, tempering, and theoretical guarantees

FM-MCMC is fully sequential. At iteration $$\frac{\mathrm{d}}{\mathrm{d}t}\phi_t(x)=v_t(\phi_t(x)),\qquad \phi_0(x)=x,$$4, after generating particles $$\frac{\mathrm{d}}{\mathrm{d}t}\phi_t(x)=v_t(\phi_t(x)),\qquad \phi_0(x)=x,$$5 using the current kernel, the flow parameters are updated by

$$\frac{\mathrm{d}}{\mathrm{d}t}\phi_t(x)=v_t(\phi_t(x)),\qquad \phi_0(x)=x,$$6

The particle system is therefore both the sampling state and the training dataset. In the theoretical analysis, the chain distribution at iteration $$\frac{\mathrm{d}}{\mathrm{d}t}\phi_t(x)=v_t(\phi_t(x)),\qquad \phi_0(x)=x,$$7 is written as

$$\frac{\mathrm{d}}{\mathrm{d}t}\phi_t(x)=v_t(\phi_t(x)),\qquad \phi_0(x)=x,$$8

and the FM objective is estimated with $$\frac{\mathrm{d}}{\mathrm{d}t}\phi_t(x)=v_t(\phi_t(x)),\qquad \phi_0(x)=x,$$9 in place of $p_t=[\phi_t]_\sharp p_0$0:

$p_t=[\phi_t]_\sharp p_0$1

To address mode discovery in strongly multimodal settings, the method introduces adaptive tempering. For Bayesian targets of the form

$p_t=[\phi_t]_\sharp p_0$2

it targets annealed distributions

$p_t=[\phi_t]_\sharp p_0$3

The next temperature is chosen by fixing an ESS fraction $p_t=[\phi_t]_\sharp p_0$4 of the particle count $p_t=[\phi_t]_\sharp p_0$5:

$p_t=[\phi_t]_\sharp p_0$6

with

$p_t=[\phi_t]_\sharp p_0$7

Empirically, this mechanism is reported as essential for discovering separated modes (Cabezas et al., 2024).

The main theoretical guarantee is a stochastic approximation convergence result. If the step sizes satisfy the Robbins-Monro conditions

$p_t=[\phi_t]_\sharp p_0$8

and the stated regularity assumptions hold, then

$p_t=[\phi_t]_\sharp p_0$9

where $$\log p_t(x_t)=\log p_0(x)-\int_0^t \nabla\cdot v_s(x_s)\,\mathrm{d}s, \qquad x_s:=\phi_s(x).$$0 is a local minimum of $$\log p_t(x_t)=\log p_0(x)-\int_0^t \nabla\cdot v_s(x_s)\,\mathrm{d}s, \qquad x_s:=\phi_s(x).$$1 (Cabezas et al., 2024). The paper explicitly notes that this is not a global optimality guarantee, because the FM objective is nonconvex. A common misconception is that the flow-learning component removes the usual nonconvexity issues of neural transport; the stated result does not support that interpretation.

5. Empirical behavior, metrics, and observed trade-offs

The reported empirical evaluation covers two synthetic and two real-world tasks: a 4-mode 2D Gaussian mixture, a 16-mode Gaussian mixture, a 64-dimensional Allen-Cahn field system, and a log-Gaussian Cox point process on a $$\log p_t(x_t)=\log p_0(x)-\int_0^t \nabla\cdot v_s(x_s)\,\mathrm{d}s, \qquad x_s:=\phi_s(x).$$2 grid with latent dimension $$\log p_t(x_t)=\log p_0(x)-\int_0^t \nabla\cdot v_s(x_s)\,\mathrm{d}s, \qquad x_s:=\phi_s(x).$$3 (Cabezas et al., 2024). The paper uses Maximum Mean Discrepancy, Kernel Stein Discrepancy with inverse multiquadratic kernel

$$\log p_t(x_t)=\log p_0(x)-\int_0^t \nabla\cdot v_s(x_s)\,\mathrm{d}s, \qquad x_s:=\phi_s(x).$$4

Monte Carlo estimates of $$\log p_t(x_t)=\log p_0(x)-\int_0^t \nabla\cdot v_s(x_s)\,\mathrm{d}s, \qquad x_s:=\phi_s(x).$$5, and wall-clock runtime.

For the 4-mode Gaussian mixture, MFM and DDS are reported as the only methods that recover all separated modes, while NF-MCMC struggles more. For the 16-mode mixture, MFM captures the full target distribution, while DDS can slightly outperform it in MMD but at much higher runtime. For the Allen-Cahn field system, adaptive tempering is described as crucial: it prevents collapse to a single mode and allows the sampler to find both global minima. On that problem, MFM is reported to be significantly faster than DDS, FAB, and NF-MCMC, often by about $$\log p_t(x_t)=\log p_0(x)-\int_0^t \nabla\cdot v_s(x_s)\,\mathrm{d}s, \qquad x_s:=\phi_s(x).$$6, while still producing strong sample quality. For the log-Gaussian Cox process, DDS achieves the best approximation quality by a small margin, but MFM is competitive and faster than DDS and FAB (Cabezas et al., 2024).

These observations define the method’s empirical profile. Its strengths are strong computational efficiency, improved global exploration over purely local MCMC, the ability to handle multimodal problems via tempering, and adaptive training directly from chain samples. Its limitations are also explicit: the FM objective is nonconvex, no non-asymptotic rates are given, and performance can depend significantly on architecture and hyperparameters. In some cases DDS can outperform MFM in pure sample quality, though often at much higher cost (Cabezas et al., 2024).

6. Relation to adjacent flow–MCMC hybrids

FM-MCMC belongs to a broader family of methods that combine transport maps with Monte Carlo correction, but the details of that combination vary substantially.

"Annealed Langevin Monte Carlo for Flow ODE Sampling" (Huang, 21 Apr 2026) is closely related but not identical. There, the method is a two-stage procedure: an annealed Langevin chain first produces weighted particles across intermediate distributions, and those particles are then used to estimate the velocity field of a probability-flow ODE derived from stochastic interpolants. The reweighting is central and Jarzynski/AIS-like, with accumulated log-weight

$$\log p_t(x_t)=\log p_0(x)-\int_0^t \nabla\cdot v_s(x_s)\,\mathrm{d}s, \qquad x_s:=\phi_s(x).$$7

and estimator identity

$$\log p_t(x_t)=\log p_0(x)-\int_0^t \nabla\cdot v_s(x_s)\,\mathrm{d}s, \qquad x_s:=\phi_s(x).$$8

The paper proves an $$\log p_t(x_t)=\log p_0(x)-\int_0^t \nabla\cdot v_s(x_s)\,\mathrm{d}s, \qquad x_s:=\phi_s(x).$$9 mean-squared error bound for the resulting velocity-field estimator and shows strong performance on multimodal targets. The data explicitly characterize this work as a strongly related supporting reference rather than the canonical source for FM-MCMC (Huang, 21 Apr 2026).

"Sampling via Föllmer Flow" (Ding et al., 2023) offers another adjacent pattern: a deterministic Gaussian-to-target ODE transport, Euler discretization, Monte Carlo approximation of the velocity, and a warm-start strategy for existing MCMC methods. Its hybrid scheme is explicitly predictor-corrector in spirit: the flow predicts a mode-covering initialization, and MCMC corrects and refines the samples. This is conceptually close to FM-MCMC, but it does not learn a flow-matching model from data in the modern neural sense (Ding et al., 2023).

"MetFlow" (Thin et al., 2020) predates flow matching as the organizing principle, but it also couples learned flow-based transformations with Metropolis-Hastings accept/reject steps. Its marginal after $Q$0 transitions becomes a mixture over many accept/reject flow paths, which the paper emphasizes as a source of expressivity. Relative to FM-MCMC, this suggests an earlier line of hybridization in which exactness comes from Metropolization and flexibility from learned invertible proposals (Thin et al., 2020).

7. Extensions, domain-specific variants, and scope of the term

The term FM-MCMC is also used more loosely to describe methods that position flow matching as an alternative to, or replacement for, conventional Markov chains in particular application domains. "Flow Matching at Scale: A Machine Learning Framework for Efficient Large-Size Sampling of Many-Body Systems" (Lee et al., 21 Aug 2025) proposes a Flow-Matching-based sampler for the 2D XY model that learns a continuous, temperature-conditioned transport map from Gaussian noise to equilibrium configurations. The interpolation path is

$Q$1

the probability-flow ODE is

$Q$2

and the training objective is a mean-squared regression on vector fields. In that work, the learned ODE is used as a deterministic sampler trained from MCMC data but then employed to bypass Markov-chain autocorrelation. The paper frames this as a practical alternative to conventional MCMC rather than as an MCMC kernel with Metropolized flow proposals (Lee et al., 21 Aug 2025).

That distinction matters for terminology. In the strict sense established by (Cabezas et al., 2024), FM-MCMC refers to an algorithm in which flow matching trains a CNF that is embedded inside a valid MCMC chain. In the broader sense used across adjacent literature, the label can also refer to flow-based transport methods designed to assist, warm-start, stabilize, or replace Markov chains. A plausible implication is that the term currently names both a specific algorithmic construction and a wider design pattern centered on combining learned transport with Monte Carlo robustness.

Within that broader landscape, the most stable technical core is the following: a reference distribution is transported along a learned or analytically derived flow; the transport is either corrected, refined, or complemented by Markovian updates; and multimodality is handled through non-local proposals, annealing, or importance reweighting rather than by relying exclusively on local random-walk dynamics (Cabezas et al., 2024).