One-Way Flow Adversarial Likelihood Models

• One-way flow adversarial likelihood models are generative models that combine adversarial training with explicit likelihood estimation while relaxing strict invertibility requirements.
• They overcome traditional architectural limitations by employing upsampling and forward Jacobian estimation techniques, resulting in improved computational efficiency and robust density computation.
• Hybrid training objectives blend adversarial and likelihood components to enhance sample fidelity, ensure complete mode coverage, and bolster adversarial robustness.

One-way flow adversarial likelihood models are a class of generative models that combine adversarial training—central to Generative Adversarial Networks (GANs)—with explicit likelihood estimation, while relaxing the architectural restrictions imposed by invertible, bijective flows. These models circumvent the inefficiencies of requiring a tractable inverse and fixed dimensionality matching between latent and data space, enabling more expressive architectures and lower computational overhead. Their theoretical and empirical developments connect adversarial density modeling, energy-based learning, hybrid likelihood-adversarial objectives, and advances in generative robustness.

1. Architectural Foundations of One-Way Flows

Traditional normalizing flows require invertible mappings, $G: \mathbb R^n \rightarrow \mathbb R^n$, with a tractable analytic inverse $G^{-1}$. The induced density is defined using the change of variables formula,

$$p_G(x) = p_Z(G^{-1}(x)) \ \left| \det \left( \frac{\partial G^{-1}(x)}{\partial x^\top} \right) \right|.$$

This imposes both dimension matching ($n = d$) and significant constraints on the network's structure (e.g., coupling layers or permutation-invariant blocks), which can preclude the use of many state-of-the-art generator architectures.

One-way flow models relax these constraints by dispensing with the need for a tractable inverse. The generator $G_\psi(z)$ is constructed as follows:

• A lower-dimensional latent variable $z \in \mathbb R^d$ ($d \leq n$) is first mapped into a higher-dimensional space via an upsampling layer $g_u: \mathbb R^d \to \mathbb R^n$. This may involve concatenating $z$ with an auxiliary random vector $r \sim p(r)$,

$G^{-1}$0

• Subsequent expansion through arbitrary, differentiable networks $G^{-1}$1 (e.g., deconvolutions, residual blocks), forming the composite generator $G^{-1}$2.

The key property is that $G^{-1}$3 need only be differentiable (not invertible), as density computations rely only on the forward Jacobian with respect to $G^{-1}$4: $G^{-1}$5 Estimating the Jacobian determinant is handled via Hutchinson-style estimators or, where feasible, tractable expansions for triangular Jacobians (Ben-Dov et al., 2023).

2. Likelihood Computation and Change-of-Variables

One-way flow models admit explicit density evaluation due to the tractable computation of the Jacobian of the forward mapping $G^{-1}$6. The induced sample density for $G^{-1}$7 is: $G^{-1}$8 where $G^{-1}$9 is the Dirac delta function and $$p_G(x) = p_Z(G^{-1}(x)) \ \left| \det \left( \frac{\partial G^{-1}(x)}{\partial x^\top} \right) \right|.$$0 denotes the $$p_G(x) = p_Z(G^{-1}(x)) \ \left| \det \left( \frac{\partial G^{-1}(x)}{\partial x^\top} \right) \right|.$$1 Jacobian matrix. Even though $$p_G(x) = p_Z(G^{-1}(x)) \ \left| \det \left( \frac{\partial G^{-1}(x)}{\partial x^\top} \right) \right|.$$2 is non-invertible, any $$p_G(x) = p_Z(G^{-1}(x)) \ \left| \det \left( \frac{\partial G^{-1}(x)}{\partial x^\top} \right) \right|.$$3 in the range of $$p_G(x) = p_Z(G^{-1}(x)) \ \left| \det \left( \frac{\partial G^{-1}(x)}{\partial x^\top} \right) \right|.$$4 can be assigned a density using the change-of-variables formulation.

Empirically, direct inversion is not required. Given generated pairs $$p_G(x) = p_Z(G^{-1}(x)) \ \left| \det \left( \frac{\partial G^{-1}(x)}{\partial x^\top} \right) \right|.$$5, the density $$p_G(x) = p_Z(G^{-1}(x)) \ \left| \det \left( \frac{\partial G^{-1}(x)}{\partial x^\top} \right) \right|.$$6 is computed using samples from the prior and evaluated Jacobian determinants. Practical estimation for high-dimensional Jacobians relies on stochastic trace estimation and Hutchinson methods, without necessitating enumeration over all outputs (Ben-Dov et al., 2023, Lucas et al., 2019).

3. Adversarial Likelihood Objectives

Adversarial likelihood models are constructed within energy-based frameworks. The fundamental distribution has the form: $$p_G(x) = p_Z(G^{-1}(x)) \ \left| \det \left( \frac{\partial G^{-1}(x)}{\partial x^\top} \right) \right|.$$7 where $$p_G(x) = p_Z(G^{-1}(x)) \ \left| \det \left( \frac{\partial G^{-1}(x)}{\partial x^\top} \right) \right|.$$8 acts as a neural energy ("critic" or unnormalized log-density). Discriminator training amounts to maximizing the data log-likelihood,

$$p_G(x) = p_Z(G^{-1}(x)) \ \left| \det \left( \frac{\partial G^{-1}(x)}{\partial x^\top} \right) \right|.$$9

The partition function $n = d$0 is intractable for real data. One-way flow models exploit importance sampling, drawing $n = d$1, $n = d$2, and evaluating

$n = d$3

This unbiased estimator stands in contrast to the biased estimator employed by one-sample WGAN methods, which neglect the generator density and thus fail to recover the proper likelihood scaling (Ben-Dov et al., 2023).

Generator training is achieved by minimizing the $n = d$4 divergence, which yields a maximization of the generator entropy $n = d$5 and the critic output on generated samples: $n = d$6 Incorporating the entropy term mitigates mode-collapse by incentivizing diversity in generated samples.

4. Hybrid and Variational Extensions

Hybrid adversarial–likelihood objectives leverage both sample synthesis quality and coverage of the full data distribution. Flow-GAN (grover et al., 2017) and Adaptive Density Estimation (ADE) (Lucas et al., 2019) exemplify this approach:

• A weighted combination of adversarial (minimax) and maximum-likelihood terms is optimized,

$n = d$7

• In variational hybrid models, an invertible mapping $n = d$8 is inserted into the decoder; the density in input space is then evaluated as

$n = d$9

where $G_\psi(z)$0 is the usual evidence lower bound in feature space.

Hybrid training empirically interpolates between pure likelihood (MLE) and adversarial (ADV) regimes: MLE-trained models show strong likelihood scores but poor sample realism, while pure adversarial models generate high-fidelity samples but exhibit low support coverage and poor likelihoods. The hybrid objective achieves a Pareto-optimal balance, yielding both high held-out likelihood and convincing sample quality (see Section 6) (grover et al., 2017, Lucas et al., 2019).

5. Robustness, Limitations, and Adversarial Conditioning

Traditional invertible flows are highly vulnerable to adversarial perturbations due to their reliance on global invertibility and the explicit inversion required for density estimation. Empirical attacks targeting both in-distribution and out-of-distribution likelihoods can sharply degrade the nominal density estimation, with NLL on adversarially perturbed images rising several orders of magnitude over clean images (Pope et al., 2019). Hybrid adversarial training, which incorporates adversarially perturbed samples into the training objective, significantly improves the robustness of the learned models.

One-way flows, by obviating the need for analytic inversion and enabling architectural surjectivity or non-invertibility, offer avenues for adversarially robust density modeling. By restricting the target of adversarial attacks to the forward mapping and Jacobian, these models can, in principle, "darken" vulnerability to perturbations exploiting the backward mapping and make generative likelihoods more stable under attack (Pope et al., 2019). A plausible implication is that one-way flow models can be further fortified for robustness using contractive or monotonic architectures, stochastic Jacobian surrogates, and compositional hybrid training regimes.

6. Empirical Results and Comparative Analysis

Empirical evaluations span synthetic and real-image datasets. Key findings include:

• On synthetic multi-modal distributions, one-way flow adversarial models recover all modes and achieve higher proportions of "high-quality" samples versus GAN baselines, with mode coverage rates of 100% in tested settings (Ben-Dov et al., 2023).
• On CIFAR-10 and CelebA, one-way flow adversarial likelihood models converge faster (as measured by FID curves), produce comparable or improved sample quality (FID of 22.5 on CelebA and 42.4 on CIFAR-10), and display minimal overfitting according to density histograms—test and train scores largely overlap (Ben-Dov et al., 2023).
• Hybrid models (as in Flow-GAN and ADE) achieve negative log-likelihood and sample fidelity competitive with the respective best explicit (e.g., Real-NVP) and implicit (e.g., DCGAN) models. For example, in (grover et al., 2017), the hybrid model obtains test NLL ≈ 4.21 bits/dim and Inception score 3.90 on CIFAR-10, bridging the performance gap between pure likelihood and adversarial models.
• In adversarial robustness studies, hybrid adversarially trained models maintain NLL under attack (≈5.9 on CIFAR-10) comparable to adversarial-only training, while preserving nominal NLL close to the clean model (≈3.6), thus nearly resolving the standard robustness–accuracy trade-off (Pope et al., 2019).

Model/Setting	NLL (bits/dim)	Inception (IS) / FID	Mode Coverage
One-way flow + ADV (Ben-Dov et al., 2023)	3.7–4.9	IS 8.1–8.5 / FID 17–45	100% (synthetic GMM)
Flow-GAN Hybrid (grover et al., 2017)	4.2	IS 5.8	High
Pure MLE (Real-NVP, Glow)	3.5–3.8	IS 3.8–5.8	Variable
Pure GAN	—	IS 8.2 / FID 21.7	Mode Dropping

7. Outlook and Open Problems

The relaxation of invertibility constraints in one-way flow models opens multiple research directions:

• One-way flows support the use of expressive, scalable generator architectures used in leading GANs, and facilitate up- and down-sampling—capabilities difficult for strictly bijective flows.
• Explicit likelihood estimation enables principled partition function estimation and generator entropy maximization, improving mode coverage and mitigating mode collapse (Ben-Dov et al., 2023).
• Hybrid models resolve the empirical tension between sample fidelity and density estimation, addressing limitations of both GANs (no density) and flows (poor samples).
• Potential further improvements may arise from adaptive or annealed weighting of hybrid objectives, better regularization of the Jacobian, or incorporation of more expressive divergences (e.g., $G_\psi(z)$1-divergences, MMD) (grover et al., 2017, Lucas et al., 2019).
• Adversarial robustness remains a dynamic area of investigation. One-way flows inherently restrict some of the adversarial attack surface, but care must be taken to avoid new vulnerabilities introduced by architectural choices. The empirical roadmap includes further development of partial invertibility, contractive networks, and adversarially-aware hybrid training (Pope et al., 2019).

Future research is guided by the theoretical and empirical successes documented above, and ongoing efforts now focus on scaling models to high-resolution domains, refining adversarial robustness, and extending the hybrid likelihood–adversarial paradigm to tasks beyond generative modeling.