Conditional Entropic FM Objective

• Conditional Entropic FM Objective is a framework for conditional generative modeling that balances data fidelity and smoothness using entropic regularization in optimal transport.
• It leverages minimax neural training and convex dual formulations to enhance local Lipschitz regularity and robustness in learning conditional maps.
• Applications include risk-sensitive flow matching and collaborative filtering, enabling efficient nonparametric estimators and scalable stochastic optimization.

The conditional entropic Fenchel–Moreau objective, or conditional entropic FM objective, refers to a suite of regularized optimization frameworks for conditional generative modeling, grounded in optimal transport (OT) and entropy-based risk. These objectives balance data fidelity and distributional smoothness when learning conditional maps, typically using minimax or convex dual formulations coupled with entropic regularization. In recent developments, this objective is realized in contexts such as conditional distribution learning with neural optimal transport, risk-sensitive flow matching, nonparametric estimators for conditional Brenier maps, and quadratic-entropy regularized linear models in collaborative filtering.

1. Foundational Formulations: Conditional Entropic Optimal Transport

Conditional entropic FM objectives are deeply rooted in the entropic-regularized OT theory. The prototypical setup considers covariate-response pairs $(x, y)$, where the goal is to learn a family of conditional distributions or generative maps $T_\theta(x, \cdot)$, parameterized by neural networks. For two covariates $x_i, x_j$, define pushforward laws

$$P_{\theta,i} = T_\theta(x_i, \cdot)_\# \mathcal{U}(0,1), \quad P_{\theta,j} = T_\theta(x_j, \cdot)_\# \mathcal{U}(0,1),$$

and compare them using the entropic $2$-Wasserstein distance ($\varepsilon$-regularized): $$W_{2,\varepsilon}^2(P_{\theta,i},P_{\theta,j}) = \min_{\pi \in \Pi(P_{\theta,i},P_{\theta,j})} \int |y - y'|^2\,d\pi(y, y') - \varepsilon H(\pi),$$ where $H(\pi)$ denotes the entropy of the coupling $\pi$. The Fenchel–Moreau (semi-dual) form expresses this as a maximization over a Kantorovich potential $v$ via Sinkhorn duality: $$W_{2,\varepsilon}^2(P_{\theta,i},P_{\theta,j}) = \max_{v\in C(\mathbb{R})} \left\{ \int v(y) dP_{\theta,i}(y) + \int v^{c,\varepsilon}(y') dP_{\theta,j}(y') \right\},$$ with the smoothed $c$-transform

$$v^{c,\varepsilon}(y') = -\varepsilon \log \int\exp\left( \frac{v(y) - |y - y'|^2}{\varepsilon} \right) dP_{\theta,i}(y).$$

This dual objective enables scalable stochastic optimization and is central for neural OT-based conditional generative modeling (Nguyen et al., 4 Jun 2024).

2. Minimax Neural Objectives and Regularization

The operational form in neural generative conditional modeling integrates the entropic FM objective into a minimax training scheme. The generator $T_\theta$ is tasked with matching empirical conditional distributions, measured via a fit term in CDF space: $\mathrm{Fit}(\theta) = \E_{i} \E_{U \sim \mathcal{U}(0,1)} \left[ | U - \hat F_{x_i}(T_\theta(x_i, U)) |^2 \right],$ where $\hat F_{x_i}$ is a kernel density estimator (KDE) of the ground-truth CDF.

To ensure local regularity and control overfitting, a graph-structured regularizer is imposed, defined by a set of sparse neighbor pairs $(i, j) \in \mathcal{E}$ (e.g., MST edges in covariate space): $$\mathrm{Reg}(\theta) = \max_{\phi} \sum_{(i, j) \in \mathcal{E}} \left\{ \int f_\phi(x_i, y)dP_{\theta,i}(y) + \int f_\phi^{c, \varepsilon}(x_i, y') dP_{\theta,j}(y') \right\},$$ where $f_\phi$ is the neural parameterization of the conditional Kantorovich potential and $f_\phi^{c, \varepsilon}$ its $c$-transform. The full learning criterion becomes

$$\min_\theta \max_\phi \left\{ \mathrm{Fit}(\theta) + \lambda \sum_{(i, j) \in \mathcal{E}} \mathcal{R}_{ij}(\theta, \phi) \right\}.$$

This minimax structure induces both generative fidelity to conditional marginals and local Lipschitz continuity in the generator over covariates, enforcing smoothness in distribution space rather than global parameter space (Nguyen et al., 4 Jun 2024).

3. Conditional Entropic Flow-Matching and Risk-Sensitive Losses

Entropic FM objectives also arise in risk-sensitive flow matching, where a velocity field $u_\theta^t(x)$ parameterizes flows between reference and data distributions in continuous time. The conditional entropic FM loss at location $(t, x)$ is given by

$E_\lambda(t, x) = \frac{1}{\lambda} \log \EE_{z \mid x, t} \exp\left( \lambda \| u_\theta^t(x) - U_t(x, z) \|^2 \right),$

where $U_t(x, z)$ are velocity targets induced by interpolated pairs. This loss penalizes not only mean-squared errors but also higher-moment fluctuations, emphasizing rare or ambiguous target velocities, and introduces gradient corrections: $$\nabla_{u}E_\lambda(t, x) = 2\,m_t(x) + 4\lambda\,\Sigma_t(x)m_t(x) - 2\lambda\,S_t(x) + O(\lambda^2),$$ where $\Sigma_t(x)$ is the conditional covariance, and $S_t(x)$ encodes the conditional third moment. The marginal entropic FM loss, a tractable upper bound via Jensen's inequality, is used in practice: $\mathcal{L}_\lambda(\theta) = \frac{1}{\lambda} \log \EE_{x, t, z} \exp\left( \lambda \| u_\theta^t(x) - U_t(x, z) \|^2 \right).$ This approach enhances sensitivity to distribution tails and substructure, which standard mean-squared error objectives cannot capture (Ramezani et al., 28 Nov 2025).

4. Statistical Estimation in Conditional Optimal Transport

The conditional entropic FM objective underpins non-parametric estimators for conditional Brenier maps. Consider joint measures $(X, Y) \sim \pi$ with reference $\rho$ (e.g., $\rho_1 \otimes \rho_2$) and target measure $\mu$. The entropic OT objective at population level is

$\min_{\pi \in \Pi(\rho, \mu)} \E_{(X, Y) \sim \pi} \left[ \frac{1}{2} \|A_t (X - Y) \|^2 \right] + \varepsilon \; \text{KL}(\pi \| \rho \otimes \mu),$

where $A_t$ is a cost-rescaling matrix and $\varepsilon$ controls entropic bias. In empirical settings, this leads to Sinkhorn-regularized discrete transport plans: $$\widehat{P}_{\varepsilon, t} = \argmin_{P \in \mathsf{DS}_n} \langle C_t, P \rangle + \varepsilon H(P),$$ with $C_t$ as the cost matrix and $H(P)$ the discrete entropy. The barycentric projection of the solution yields a consistent nonparametric conditional map, converging to the conditional Brenier map as the sample size increases and $\varepsilon, t \to 0$, with prescribed scaling laws: $$t(n) \asymp n^{-1/3}, \quad \varepsilon(n) \asymp n^{-2/3}$$ (Baptista et al., 11 Nov 2024).

5. Quadratic-Entropy Regularization for Conditional Linear Models

Historically, a quadratic-entropy (Rényi-2) surrogate objective has been employed in collaborative filtering as a conditional entropic regularizer. The goal is to estimate conditional probabilities $p_i(x)$ matching low-order marginal constraints while minimizing

$J(p) = \sum_x \tilde{P}(x) \, [p(x)]^2,$

subject to affine expectations over chosen binary features. The closed-form solution is linear in the features,

$p_i(x) = \sum_{k=0}^{K-1} w_{i,k} f_k(x),$

where weights $w_i$ solve the $K \times K$ linear system

$A w_i = b_i,$

with $A_{j,k} = \tilde{P}(f_j=1, f_k=1)$ and $b_{i, j} = \tilde{P}(y_i=1, f_j=1)$. This approach yields an efficient, principled solution for conditional prediction under entropic regularization, serving as both a standalone method and as a warm-start or regularizer for more expressive factorization machines (Zitnick et al., 2012).

6. Practical Optimization and Algorithmic Aspects

Algorithmic realization of conditional entropic FM objectives typically deploys stochastic gradient descent–ascent with smoothing. For neural OT-based objectives, variables $\theta$ (generator) and $\phi$ (potential) are updated via GDA on a surrogate objective incorporating quadratic prox terms for stability: $$\mathcal{L}(\theta, \phi, p, q) = \mathrm{Fit}(\theta) + \lambda \sum_{(i, j) \in \mathcal{E}} \mathcal{R}_{ij}(\theta, \phi) + \tfrac{r_1}{2} \|\theta - p\|^2 - \tfrac{r_2}{2} \|\phi - q\|^2.$$ Estimators for regularization terms and functionals are produced via Monte Carlo sampling. Hyperparameters—entropic weight, regularization strength, smoothing, and stepsizes—are tuned by cross-validation on suitable metrics (e.g., Wasserstein or KS distances). For discrete OT-based estimators, computational costs scale with Sinkhorn algorithm iterations and benefit from GPU or kernel methods for large $n$ (Nguyen et al., 4 Jun 2024, Baptista et al., 11 Nov 2024).

7. Empirical Impact and Applications

In synthetic and real-world experiments, conditional entropic FM objectives have demonstrated the ability to recover fine-grained geometric, marginal, and tail structures in conditional generative modeling. Risk-sensitive entropic flow-matching loss improves angular spread and gap-violation rate in ambiguous transport settings. Neural entropic OT-based conditional generators achieve improved generalization and stability under limited sample regimes and outperform state-of-the-art competitive baselines. Closed-form quadratic-entropy approaches enable efficient large-scale collaborative filtering and inform regularization strategies for more expressive models (Zitnick et al., 2012, Nguyen et al., 4 Jun 2024, Ramezani et al., 28 Nov 2025). A plausible implication is the broad versatility of entropic FM objectives across both deep learning and classical conditional modeling contexts.