Conditional Generator Matching Loss

• Conditional Generator Matching Loss is a framework that optimizes generative models by aligning a learned conditional generator's output with the target distribution using metrics like the Wasserstein distance.
• It employs mathematical formulations such as minimax, dual formulations, and Bregman divergences to ensure robust optimization and reliable error bounds in high-dimensional settings.
• The approach underpins practical applications including conditional sample generation, density estimation, and uncertainty quantification for tasks like image reconstruction and inverse problems.

Conditional Generator Matching Loss refers to a principled family of training objectives for generative models in which a parameterized generator function is optimized to match conditional (often noise-to-data) distributions, typically via adversarial, flow-matching, or score-matching objectives. Its variants include the Wasserstein Conditional Sampler loss, conditional generator/flow matching loss for Markov processes, and recent formulations for conditional score-based and flow-matching generative modeling. These losses underpin state-of-the-art approaches for conditional sample generation, conditional density estimation, uncertainty quantification, and high-dimensional generative modeling.

1. Mathematical Formulation

The core structure of Conditional Generator Matching Loss is to match a learned conditional generator's output distribution to a target conditional (or joint) distribution, typically using distances or divergences amenable to optimization.

• In Wasserstein Conditional Sampling (Liu et al., 2021), for observed data pairs $(X, Y)\sim P_{X,Y}$, and latent variable $\eta\sim P_\eta$,

$$G^* = \arg\min_{G} W_1\big(P_{X,G(\eta,X)},P_{X,Y}\big),$$

where $W_1$ is the 1-Wasserstein distance on $\mathbb{R}^{d+q}$, and $$G:\mathbb{R}^m\times\mathbb{R}^d\rightarrow\mathbb{R}^q$$ is a generator such that $G(\eta,x)\sim P_{Y|X=x}$.

• Via Kantorovich–Rubinstein duality, this loss admits a minimax formulation,

$$\min_G \max_{D\in\mathrm{Lip}_1} \big\{ \mathbb{E}_{(X,\eta)} D(X,G(\eta,X)) - \mathbb{E}_{(X,Y)} D(X,Y) \big\},$$

where $D$ is a 1-Lipschitz critic.

• In generator matching for Markov processes (Holderrieth et al., 2024), the Conditional Generator Matching (CGM) loss generalizes to arbitrary Markovian paths. For a family of conditional distributions $p_t(dx|z)$ with infinitesimal generators $\eta\sim P_\eta$0 (typically known in closed form), the CGM loss with pointwise Bregman divergence $\eta\sim P_\eta$1 is

$\eta\sim P_\eta$2

where $\eta\sim P_\eta$3 is the generator for the conditional path (analytically available), and $\eta\sim P_\eta$4 is the neural approximation to the marginal generator.

• In conditional flow matching (Dasgupta et al., 14 Mar 2026), the loss for a velocity field $\eta\sim P_\eta$5 transporting a source distribution to a conditional (posterior) is

$\eta\sim P_\eta$6

with an interpolant $\eta\sim P_\eta$7 and samples from the joint.

• Conditional score matching losses (e.g., denoising likelihood score matching) (Chao et al., 2022) and flow matching (Bertrand et al., 4 Jun 2025) are also mathematically subsumed in this framework via specialized instantiations of $\eta\sim P_\eta$8 and the target generator $\eta\sim P_\eta$9.

2. Dual and Minimax Forms

The minimax and dual formulations are especially prominent in Wasserstein-based and adversarial generator matching.

• In Wasserstein Conditional Sampler, the maximization over 1-Lipschitz $$G^* = \arg\min_{G} W_1\big(P_{X,G(\eta,X)},P_{X,Y}\big),$$0 is implemented via a neural critic and a gradient-penalty regularization term,

$$G^* = \arg\min_{G} W_1\big(P_{X,G(\eta,X)},P_{X,Y}\big),$$1

maintaining Lipschitzness (Liu et al., 2021).

• The overall optimization is conducted by alternating gradient descent-ascent steps for $$G^* = \arg\min_{G} W_1\big(P_{X,G(\eta,X)},P_{X,Y}\big),$$2 and $$G^* = \arg\min_{G} W_1\big(P_{X,G(\eta,X)},P_{X,Y}\big),$$3: $$G:\mathbb{R}^m\times\mathbb{R}^d\rightarrow\mathbb{R}^q$$9
• In general generator matching (Holderrieth et al., 2024), the CGM loss exploits the Bregman divergence's affine property in $$G^* = \arg\min_{G} W_1\big(P_{X,G(\eta,X)},P_{X,Y}\big),$$4, so the samplewise minimization gradients coincide with those for the otherwise intractable marginal generator-matching loss.

3. Algorithmic Implementation

A common structure emerges across frameworks:

• Sampling: Draw minibatches of $$G^* = \arg\min_{G} W_1\big(P_{X,G(\eta,X)},P_{X,Y}\big),$$5 and/or latent $$G^* = \arg\min_{G} W_1\big(P_{X,G(\eta,X)},P_{X,Y}\big),$$6 or Markov process samples $$G^* = \arg\min_{G} W_1\big(P_{X,G(\eta,X)},P_{X,Y}\big),$$7.
• Generation: Form conditional samples $$G^* = \arg\min_{G} W_1\big(P_{X,G(\eta,X)},P_{X,Y}\big),$$8 or intermediary $$G^* = \arg\min_{G} W_1\big(P_{X,G(\eta,X)},P_{X,Y}\big),$$9.
• Critic/Evaluator: Compute either a 1-Lipschitz critic $W_1$0, the conditional generator target $W_1$1 (for Markov paths), or score targets.
• Gradient/Update:
  • For Wasserstein: alternate maximizing critic loss and minimizing generator loss, enforcing Lipschitz via penalties.
  • For CGM: directly regress $W_1$2 to $W_1$3 over sampled $W_1$4 points, accumulating Bregman divergences and backpropagating.
  • For flow-matching: regress neural velocity $W_1$5 or $W_1$6 to conditional velocity/score targets.
• Optimizers: Typically Adam or other stochastic gradient methods as in the sample pseudocode blocks in (Liu et al., 2021, Chao et al., 2022).

4. Theoretical Properties and Error Bounds

Conditional Generator Matching Loss enjoys rigorous non-asymptotic guarantees in various settings.

• (Liu et al., 2021): For generator/critic networks of appropriate capacity (width-depth scaling with sample size $W_1$7), it is shown

$W_1$8

under moment and compactness conditions, with extensions replacing $W_1$9 by intrinsic Minkowski dimension $\mathbb{R}^{d+q}$0 for low-dimensional support, mitigating the curse of dimensionality.

• (Holderrieth et al., 2024): The CGM loss gradient matches that of the marginal generator-matching loss, so stochastic optimization on the CGM objective yields unbiased estimates for generator parameter updates.
• (Dasgupta et al., 14 Mar 2026): Exact minimization of the conditional flow-matching loss ensures the learned flow map transports the source $\mathbb{R}^{d+q}$1 to the exact conditional $\mathbb{R}^{d+q}$2 at $\mathbb{R}^{d+q}$3. In the finite data regime, overfitting can cause degenerate behaviors: variance collapse (posterior becomes a Dirac at the empirical conditional mean) or selective memorization (posterior reduces to nearest neighbor pseudo-posterior). Early stopping based on held-out test loss effectively mitigates these failures.

5. Generalizations and Special Cases

Conditional Generator Matching Loss encompasses a wide range of modern generative modeling paradigms via specific settings of the process, generator parameterization, and divergence:

Setting	Conditional generator ($\mathbb{R}^{d+q}$4)	Discrepancy $\mathbb{R}^{d+q}$5
Score-based diffusion	Score function $\mathbb{R}^{d+q}$6	Squared $\mathbb{R}^{d+q}$7
Flow matching	Conditional velocity $\mathbb{R}^{d+q}$8	Squared $\mathbb{R}^{d+q}$9
Jump processes	Jump kernel $$G:\mathbb{R}^m\times\mathbb{R}^d\rightarrow\mathbb{R}^q$$0	KL or entropy-based
Wasserstein	Sample-to-sample map via $$G:\mathbb{R}^m\times\mathbb{R}^d\rightarrow\mathbb{R}^q$$1	Wasserstein-$$G:\mathbb{R}^m\times\mathbb{R}^d\rightarrow\mathbb{R}^q$$2

Classical denoising score matching (Chao et al., 2022) and conditional flow matching losses (Bertrand et al., 4 Jun 2025, Dasgupta et al., 14 Mar 2026) are obtained as special cases of the general CGM or Wasserstein matching frameworks.

For instance, in score-based modeling, CGM with MSE on vector fields recovers the traditional denoising score-matching loss

$$G:\mathbb{R}^m\times\mathbb{R}^d\rightarrow\mathbb{R}^q$$3

while in flow-matching, the velocity regression loss

$$G:\mathbb{R}^m\times\mathbb{R}^d\rightarrow\mathbb{R}^q$$4

serves as the generator matching objective (Dasgupta et al., 14 Mar 2026).

6. Representative Applications

Conditional Generator Matching Loss is foundational in a broad spectrum of conditional generative tasks. As documented in (Liu et al., 2021, Holderrieth et al., 2024), and related works, examples include:

• Conditional sample generation: Accurate modeling of $$G:\mathbb{R}^m\times\mathbb{R}^d\rightarrow\mathbb{R}^q$$5 in structured simulation tasks (two-moons, synthetic manifolds).
• Nonparametric conditional density estimation: Superior mean-squared error performance for $$G:\mathbb{R}^m\times\mathbb{R}^d\rightarrow\mathbb{R}^q$$6 with heteroskedastic or mixture noise structure, outperforming KDE variants.
• Uncertainty quantification and prediction intervals: For wine-quality data and bivariate regression, the conditional generator-based approach yields credible intervals with desired coverage properties.
• Inverse problems: In physics-constrained settings, the conditional flow matching approach efficiently solves for posteriors without explicit likelihood evaluation (Dasgupta et al., 14 Mar 2026).
• High-dimensional scenario: In image reconstruction (e.g., partial-to-whole MNIST digits), attribute-guided face generation (CelebA), and large-scale flow matching (CIFAR-10, CelebA datasets), generator matching losses yield high-quality, semantically accurate, and diverse outputs.
• Analysis of generalization: Empirical studies (Bertrand et al., 4 Jun 2025) demonstrate that in high dimensions, conditional flow matching’s stochastic target can be replaced by closed-form (deterministic) regression without performance penalty, validating the mathematical structure of the underlying loss.

7. Connections, Limitations, and Regularization

Conditional Generator Matching Loss unifies adversarial, flow-based, and score-based training through the lens of infinitesimal generator matching. The following considerations are essential:

• All methods require the tractability of the conditional generator target ($$G:\mathbb{R}^m\times\mathbb{R}^d\rightarrow\mathbb{R}^q$$7 or equivalents) and efficient sampling from the corresponding Markov or noise processes.
• Regularization is typically needed for valid generator parameterization (e.g., positivity for diffusions/jump kernels, eigenvalue constraints), and for enforcing Lipschitz continuity in Wasserstein settings.
• Failure modes in limited data or overparameterized regimes (variance collapse, memorization) necessitate monitoring (e.g., early stopping on test loss).
• Choice of pointwise divergence ($$G:\mathbb{R}^m\times\mathbb{R}^d\rightarrow\mathbb{R}^q$$8, KL, etc.) and process (diffusion, flow, jump) determines the expressiveness and statistical behavior.

Conditional Generator Matching Loss thus provides a mathematically principled, empirically robust foundation for modern conditional generative modeling, seamlessly spanning adversarial, flow-based, and score-matching paradigms (Liu et al., 2021, Holderrieth et al., 2024, Chao et al., 2022, Bertrand et al., 4 Jun 2025, Dasgupta et al., 14 Mar 2026).