Bayesian CycleGANs: A Probabilistic Framework

• Bayesian CycleGANs are generative adversarial networks enhanced by a Bayesian framework that employs explicit Gaussian priors, MAP estimation, and latent variable marginalization to mitigate mode collapse.
• Their dual-generator architecture and integrated cyclic losses enable stable inter-domain mapping by enforcing adversarial constraints on both direct and reconstructed outputs.
• Empirical evaluations demonstrate significant improvements in per-pixel accuracy and output diversity, outperforming classical CycleGAN models on benchmarks like Cityscapes.

Bayesian CycleGANs implement cycle-consistent generative adversarial networks in a full Bayesian framework, combining maximum a posteriori (MAP) estimation with latent variable marginalization and providing mechanisms for mitigating mode collapse and enabling output diversity. These models formalize inter-domain mapping with unpaired datasets as inference in latent variable models, exploring the full posteriors over generator and discriminator weights as well as latent inputs. Bayesian CycleGANs incorporate explicit Gaussian priors, augment adversarial objectives with latent-conditioned and reconstruction-based losses, and leverage customized training protocols to achieve improved stability and diversity relative to classical CycleGANs (You et al., 2018, Tiao et al., 2018).

1. Model Structure and Integrated Cyclic Losses

Bayesian CycleGAN employs a dual-generator architecture, $G_A: X \to Y$ and $G_B: Y \to X$, with each generator parameterized as a ResNet comprising two convolutional downsampling layers, six residual blocks, two upsampling layers, and instance normalization. Two Patch-GAN discriminators, $D_A$ and $D_B$, respectively classify real versus generated samples in each domain.

An integrated cyclic framework augments the standard adversarial losses: in addition to adversarial objectives on direct generated samples ($\tilde y = G_A(x)$ vs.\ real $y$), adversarial losses are placed on reconstructions ($\hat y = G_A(G_B(y))$ vs.\ real $y$), controlled by a tunable balance factor $\gamma$. This bipartite structure explicitly involves the reconstructed cycle-mappings in the adversarial dynamics, in contrast to original CycleGAN which applies adversarial loss only to the direct generative mapping and cycle-consistency only as an $L_1$ reconstruction term (You et al., 2018).

2. Bayesian Formulation and Posterior Objectives

The Bayesian framework assigns Gaussian priors to all network parameters: $$p(\theta_{ga}),\, p(\theta_{gb}),\, p(\theta_{da}),\, p(\theta_{db}) \sim \mathcal{N}(0, \alpha^{-1}I)$$ and standard normal priors to per-sample latent variables $z \sim \mathcal{N}(0, I)$. For advanced diversity, "statistic feature maps" (SFMs) drawn from a lightweight VAE-style encoder regularized by $\mathrm{KL}(q(f)\,\|\,\mathcal{N}(0,I))$ can be introduced during training.

The full posteriors over parameters, with latent variable marginalization, yield generator and discriminator objectives that integrate over sampled latent input maps. The canonical loss minimized during MAP estimation is: $$\mathcal{L} = \mathcal{L}_{GAN}^{A}(G_A, D_A) + \mathcal{L}_{GAN}^{B}(G_B, D_B) + \gamma [\mathcal{L}_{GAN}^{A}(G_B \circ G_A, D_A) + \mathcal{L}_{GAN}^{B}(G_A \circ G_B, D_B)] + \lambda (\mathcal{L}_{CYC}^A + \mathcal{L}_{CYC}^B) + \sum_{\phi} \|\phi\|_2^2$$ where the $\|\phi\|_2^2$ weight decay arises from the Gaussian parameter priors. Losses for $D_A$ and the joint generator $G=(G_A,G_B)$ include additional weighting by $\gamma$ and marginalization over latent variable samples (You et al., 2018).

3. Latent Variable Marginalization and Training Algorithm

Bayesian CycleGAN avoids weight-space Monte Carlo sampling (as in traditional Bayesian GANs), instead performing marginalization over latent variables by concatenating multiple noise maps ($z^{(i,k)}$) with each input sample, forming "stitched" inputs

$$x_z^{(i,k)} = [x^{(i)} ; z^{(i,k)}] \in \mathbb{R}^{(C+1)\times H\times W}$$

and similarly for $y$. The expected losses are approximated as sums over these sampled latent variable instantiations.

A typical mini-batch SGD iteration proceeds as:

1. Sample $m_x$ and $m_y$ latent maps for each $x$ and $y$.
2. Concatenate and forward through generators to produce direct and reconstructed outputs.
3. Update generator parameters (including SFM encoder if present) via gradients of the log-posterior (MAP, equivalent to negative total generator loss).
4. Update discriminators via their log-posterior gradients (negative total discriminator loss).

If using SFM, a $$\lambda_{KL} \, \mathrm{KL}(q(f)\| \mathcal{N}(0,I))$$ term is added to the generator update to encourage compact, informative latent encoding (You et al., 2018).

4. Mechanisms for Stability and Mode Collapse Mitigation

Bayesian CycleGANs stabilize training through two key mechanisms. First, latent-conditioned sampling smears generator output distributions, preventing rapid discriminator convergence and reducing pressure toward mode collapse. Second, the balance factor $\gamma$ places adversarial constraints directly on reconstructed (cycled) images, meaning that the discriminator learns from a wider pool of fakes, further discouraging collapse.

At equilibrium, the adversarial game is structured to achieve: $$p_Y = (1+\gamma)p_Y + p_{\tilde Y} + \gamma p_{\hat Y}$$ so that optimality is achieved when $p_{real} = p_{generated} = p_{reconstructed}$ for both mapped and reconstructed domains.

Empirical results on Cityscapes demonstrate resilience against collapse at high $\gamma$: for $\gamma=0.5$, vanilla CycleGAN yields identical label-maps for every input (mode collapse), while Bayesian CycleGAN maintains $>65\%$ per-pixel accuracy (CycleGAN: $\sim45\%$) (You et al., 2018).

5. Output Diversification via Latent Variables

Bayesian CycleGAN enables output diversity by restricting the latent space during training to a subspace formed by "statistic feature maps" generated from a VAE-like encoder. The strong cycle-consistency constraint empirically forces the trained model to ignore the SFM unless it is within the training subspace. At inference time, arbitrary noise or alternative SFMs can be substituted, producing visually diverse outputs for a fixed input. This is not possible in original CycleGAN due to the absence of explicit latent variable conditioning (You et al., 2018).

6. Relation to Approximate Bayesian Inference and CycleGAN Specialization

Cycle-consistent adversarial learning can be recovered as a special case of variational Bayesian inference in implicit latent variable models. Introducing an implicit prior over the latent space (e.g., drawn empirically from the target domain), and employing a symmetric KL optimization between a generative joint model $p_\theta(x, z)$ and variational joint $q_\phi(x, z)$, the adversarial and cycle-consistency losses arise as specializations:

• The adversarial losses correspond to lower bounds on density ratio KL-divergences.
• Cycle-consistency losses are interpreted as log-likelihood terms in degenerate (zero-variance) Gaussian models.
• The relative weighting of generative and cycle-consistency terms corresponds to the variances in these approximations (Tiao et al., 2018).

Consequently, Bayesian CycleGANs provide a principled probabilistic semantics for each network component, and the adversarial and cycle-consistency objectives become interpretable in terms of variational inference with sample-based implicit priors. This synthesis suggests flexible extensions, such as learned hyperpriors or robust cycle-consistency costs (Tiao et al., 2018).

7. Experimental Evaluation and Comparative Results

Empirical studies demonstrate consistent performance improvements over both original and "integrated" CycleGANs. Selected results include:

• On Cityscapes, Bayesian CycleGAN elevates per-pixel accuracy from $48\%$ (CycleGAN, $\gamma=0$) to $73\%$ (Bayesian, $\gamma=0$). Under integrated losses ($\gamma=0.5$), Bayesian achieves $65\%$ versus CycleGAN's $45\%$ (mode collapse regime).
• On Maps $\leftrightarrow$ Aerial, Bayesian CycleGAN yields FID (Aerial) $71.6 \to 68.5$, and Inception score $3.50 \to 3.68$ at $\gamma=0$.
• On Monet2Photo, Inception scores increase in both directions (e.g., Monet$\to$Photo from $3.76 \to 3.86$).
• Ablation analyses indicate clear superiority of Bayesian latent marginalization over Dropout and buffer heuristics for stabilizing training and enabling diversity (You et al., 2018).

These results confirm that Bayesian CycleGANs offer improved resilience to adversarial imbalance, maintain stability under challenging regime, and enable sample diversity through explicit latent variable modeling.

---

Key References:

• "Bayesian Cycle-Consistent Generative Adversarial Networks via Marginalizing Latent Sampling" (You et al., 2018)
• "Cycle-Consistent Adversarial Learning as Approximate Bayesian Inference" (Tiao et al., 2018)