Conditional Generator using MMD (CGMMD)

• The paper introduces CGMMD with direct ECMMD minimization, enabling one-shot conditional sampling that outperforms adversarial methods in stability and computational efficiency.
• CGMMD leverages kernel mean embeddings and nearest-neighbor estimators to match empirical conditional distributions accurately across tasks like image super-resolution and predictive modeling.
• The method offers robust theoretical guarantees with finite-sample error bounds and asymptotic convergence, providing a scalable alternative to traditional adversarial and transport-based approaches.

Conditional Generator using MMD (CGMMD) refers to a family of model architectures and training algorithms for conditional generative modeling that leverage maximum mean discrepancy (MMD) as a statistical criterion for matching conditional distributions, enabling adversary-free, statistically grounded, and computationally efficient one-shot conditional sampling. CGMMD frameworks are distinguished by their use of kernel mean embeddings to compare empirical conditional distributions and samples from a parameterized conditional generator, with direct minimization of an extended MMD loss (or its operator variants) replacing minimax adversarial games, iterative refinement, or likelihood maximization. Recent works have developed rigorous finite-sample and asymptotic convergence guarantees, efficient empirical estimators based on nearest-neighbor graphs, and demonstrated competitive real-world performance on structured synthesis tasks, including conditional image denoising and super-resolution (Chatterjee et al., 29 Sep 2025), predictive modeling (Ren et al., 2016), and Bayesian posterior sampling (Hagemann et al., 2023). CGMMD is deeply connected to the broader landscape of moment-matching, optimal transport, and gradient-flow generative modeling, and provides a flexible statistical machinery for direct matching of conditional laws with explicit generalization rates.

1. Mathematical Foundations and Conditional MMD Criteria

CGMMD generalizes the classical unconditional MMD two-sample test to the conditional setting. Given random variables $(Y, X)$, the objective is to learn a generator $g(\eta, x)$ such that the conditional law of $g(\eta, x)$ (where $\eta$ is an independent noise variable) matches $P(Y|X=x)$. The core statistical criterion is the Expected Conditional Maximum Mean Discrepancy (ECMMD):

$$\text{ECMMD}^2(\mathcal{F}_K, P_{Y|X}, P_{g(\eta, X)|X}) = \mathbb{E}_X\left[ \text{MMD}^2(\mathcal{F}_K, P(Y|X=x), P(g(\eta, x)|x)) \right]$$

where $\mathcal{F}_K$ is an RKHS associated with kernel $K$. The generator is trained to minimize ECMMD, which vanishes if and only if the conditional distributions coincide almost everywhere (Chatterjee et al., 29 Sep 2025, Ren et al., 2016). Recent operator-theoretic extensions employ conditional embedding operators $C_{Y|X}$ and empirical kernel matrices to estimate the conditional MMD criterion for more structured data (Ren et al., 2016). Empirical estimators leverage nearest-neighbor graphs in predictor space to construct sample-based conditional MMD (Chatterjee et al., 29 Sep 2025), providing unbiased and computationally efficient losses.

2. CGMMD Architectures and Training Methodologies

CGMMD instantiates conditional generators as parameterized neural networks $g_\theta(\eta, x)$ from a functional class $\mathcal{G}$. Training proceeds by direct minimization of the empirical ECMMD loss over a dataset $\{(Y_i, X_i)\}_{i=1}^n$. For each sample, neighbors in $X$ are identified, and the generator’s output is compared to the local conditional distribution via the kernel mean map. The symmetric kernel function $H$ aggregates pairwise discrepancies:

$$\widehat{L}(g) = \frac{1}{n k_n} \sum_{i=1}^n \sum_{j \in N(X_i)} H(W_i, W_j)$$

with $W_i = (Y_i, g(\eta_i, X_i))$ and $N(X_i)$ the k-nearest neighbors of $X_i$. This loss is minimized over $\theta$ without adversarial optimization, yielding one-shot conditional sampling. Previous architectures include deep multilayer perceptrons (Li et al., 2015), CNNs, and autoencoder-augmented generators (Ren et al., 2016); the choice of generator network is prescribed by the modeling goal and data domain. For sample efficiency, minibatch training is favored; efficient Gram matrix computation is enabled by restricting loss computation to neighborhood graphs or leveraging kernel slicing (Hertrich et al., 2023).

3. Theoretical Guarantees: Uniform Concentration, Error Bounds, and Convergence

Rigorous generalization bounds underpin CGMMD’s statistical foundation. Uniform concentration inequalities over function classes $\mathcal{G}$ ensure that the empirical ECMMD estimator uniformly approximates the population loss; bounds of the form

$$\sup_{g \in \mathcal{G}} |\widehat{L}(g) - L(g)| \leq C_1^* \mathbb{E}[G_m(\mathcal{G}(\mathbf{X}))] + C_2^* \sqrt{\frac{\log(2/\delta)}{n}}$$

have been established for kernel statistics (Ni et al., 22 May 2024, Chatterjee et al., 29 Sep 2025), where $G_m$ denotes empirical Gaussian complexity. For CGMMD employing nearest neighbor estimators, additional terms account for graph geometry (node degrees/empirical complexity) (Chatterjee et al., 29 Sep 2025). Main theorems show finite-sample error bounds for the learned generator, scaling as polylogarithmic in $n$ with modulus of continuity and approximation error contributions. Asymptotically, sample-based CGMMD recovers the true conditional law in kernel-mean embedding, and in distribution under mild regularity (Chatterjee et al., 29 Sep 2025, Ren et al., 2016).

4. Applications: Structured Conditional Sampling and Posterior Inference

CGMMD has been validated on diverse applications:

• Synthetic conditional density estimation: On bivariate tasks (e.g., helix, circular conditional densities), CGMMD preserves complex conditional structure better than adversarial or diffusion-based baselines (Chatterjee et al., 29 Sep 2025).
• Image super-resolution and denoising: CGMMD produces high-fidelity reconstructions in conditional MNIST and STL10 tasks, leveraging fast test-time sampling (one forward pass vs. iterative denoising in diffusion models) (Chatterjee et al., 29 Sep 2025). Empirical comparisons reveal competitive sample quality and a 100$\times$ reduction in inference time against guided diffusion generators.
• Predictive modeling: In classification on MNIST and SVHN, CGMMN (a variant of CGMMD) achieves error rates as low as $0.47\%$ with CNN architectures (Ren et al., 2016).
• Bayesian knowledge distillation: CGMMD outperforms classical GANs and matches Bayesian model predictions in uncertainty estimation (Ren et al., 2016).
• Medical imaging and missing modality imputation: Conditional U-Net architectures, equipped with multimodal correlation constraints, generate complete MRI modality sets, improving tumor segmentation accuracy (Zhou et al., 2021).
• Posterior and uncertainty quantification in inverse problems: Conditional MMD-based gradient flows with negative distance kernels offer efficient posterior sampling for superresolution, inpainting, and tomography (Hagemann et al., 2023).

5. Comparison with Adversarial and Optimal Transport Approaches

CGMMD differs fundamentally from GAN-based and optimal transport methods:

• Statistical matching vs. adversarial games: CGMMD eschews iterative min-max training; its loss is positive-definite and vanishes precisely at distributional equality (Li et al., 2015, Chatterjee et al., 29 Sep 2025), promoting stable convergence and avoiding mode collapse.
• Kernel choice and adaptation: Unlike fixed kernel approaches, advanced architectures adapt mixture kernel bandwidths and increase critic complexity during training for improved high-dimensional matching (Hofert et al., 29 Aug 2025).
• Optimal transport/geodesic generators: While Wasserstein-geodesic generators construct explicit geodesics and interpolate conditional distributions in metric space (Kim et al., 2023), CGMMD favors mean-embedding and moment matching, which is tractable and can be adapted to the conditional setting via kernel design or operator-theoretic techniques. Energy distance (MMD with negative distance kernel) and sliced Wasserstein flows are connected, enabling fast computation and gradient flow-based conditional sampling (Hagemann et al., 2023, Hertrich et al., 2023).

6. Statistical and Computational Efficiency

CGMMD achieves practical one-shot conditional sampling—requiring only a single forward pass—thus drastically reducing test-time computational cost relative to GANs or diffusion models. Nearest neighbor-based empirical estimators minimize memory usage and offer unbiased sample matching (Chatterjee et al., 29 Sep 2025). Adaptivity in kernel bandwidths (median-heuristics, empirical quantiles) and critic updates further enhance learning in high-dimensional spaces (Hofert et al., 29 Aug 2025), and sliced variants provide $O((M+N)\log(M+N))$ complexity in gradient computation (Hertrich et al., 2023). Generalization error decays as $O(n^{-1/2})$ in typical network classes, with empirical bounds matching theory (Ni et al., 22 May 2024).

7. Future Directions, Limitations, and Open Questions

CGMMD frameworks are extensible to discrete conditioning variables, multimodal data, and structured domains (graphs, sequences). Potential research avenues include:

• Richer generator architectures and regularization to enhance sample fidelity in very high-dimensional or multimodal settings.
• Theoretical analysis on non-Euclidean spaces, discrete supports, and concentration under heavy-tailed kernels or dependent data.
• Combining moment matching with transport-based objectives for improved conditional support alignment.
• Exploration of supervised and unsupervised kernel learning, permutation-based tests, and automated selection of regularization parameters for further stability and interpretability (Potapov et al., 2019, Li et al., 2017).

CGMMD provides a rigorously justified, flexible, and computationally effective class of models for conditional generative modeling, with direct minimization of statistical discrepancy, explicit finite-sample and asymptotic guarantees, and competitive empirical results in structured synthesis and inference tasks.