UGAC: Uncertainty-Aware Adaptive Cycle Consistency

• The paper introduces UGAC, which replaces deterministic cycle consistency with an uncertainty-aware, generalized Gaussian modeling approach that adapts to pixel residuals.
• Its architecture integrates cascaded U-Nets and PatchGANs to predict cyclic images, scale, and shape maps, yielding significant improvements in AMSE and ASSIM metrics.
• UGAC provides closed-form aleatoric and epistemic uncertainty measures, enhancing robustness against outlier effects and noise in diverse applications.

Uncertainty-aware Generalized Adaptive Cycle Consistency (UGAC) is a probabilistic framework for unpaired image-to-image translation that introduces pixel-wise uncertainty modeling to CycleGAN architectures using adaptive, heavy-tailed residual distributions. UGAC addresses the non-robustness of standard deterministic cycle-consistency losses under outlier or out-of-distribution test scenarios, providing calibrated per-pixel uncertainty and significantly improving test-time robustness across diverse application domains, including natural images, autonomous driving, maps, facades, and medical imaging (Upadhyay et al., 2021, Upadhyay et al., 2021).

1. Mathematical Foundation: Generalized Gaussian Cycle-Consistency

UGAC models the residual between an input pixel and its cyclic reconstruction as a zero-mean generalized Gaussian distribution (GGD), parametrized per pixel by learnable scale ($\alpha_{ij} > 0$) and shape ($\beta_{ij} > 0$) maps. The probability density function is

$$\mathrm{GGD}(\epsilon;\,0,\alpha,\beta) = \frac{\beta}{2\,\alpha\,\Gamma\left(\frac{1}{\beta}\right)} \exp\left(-\left|\frac{\epsilon}{\alpha}\right|^\beta\right)$$

where $\Gamma(\cdot)$ denotes the gamma function. The GGD subsumes Gaussian ($\beta=2$), Laplace ($\beta=1$), and can model heavier tails for $\beta<1$.

The negative log-likelihood–based cycle-consistency loss for image $u$ is then

$$\ell_{\alpha\beta}(\bar u, u) = \frac{1}{K} \sum_{j=1}^K \left[ \left| \frac{\bar u_{ij} - u_{ij}}{\alpha_{ij}} \right|^{\beta_{ij}} - \ln\frac{\beta_{ij}}{\alpha_{ij}} + \ln\Gamma\left(\frac{1}{\beta_{ij}}\right) \right]$$

Full UGAC cycle-consistency across domains $A$ and $B$ is

$$\mathcal{L}_{\mathrm{ucyc}} = \mathbb{E}_{a\sim A}\left[\ell_{\alpha\beta}\left(\mathcal{G}_B(\mathcal{G}_A(a)),a\right)\right] + \mathbb{E}_{b\sim B}\left[\ell_{\alpha\beta}\left(\mathcal{G}_A(\mathcal{G}_B(b)),b\right)\right]$$

This formulation down-weights high-residual or uncertain pixels adaptively via the learned $\alpha$ and $\beta$, as opposed to standard $\ell_1/\ell_2$ cycle losses which assume fixed variance and treat all pixels equally (Upadhyay et al., 2021, Upadhyay et al., 2021).

2. Full Objective and Architecture

UGAC is implemented atop a CycleGAN backbone but replaces the deterministic cycle loss with the GGD-based uncertainty-aware adaptive cycle-consistency. The overall generator and discriminator objectives include both adversarial and cycle-consistency terms: $$\mathcal{L}^G = \lambda_1\,\mathcal{L}_{\mathrm{ucyc}} + \lambda_2\,\mathcal{L}_{\mathrm{adv}}^G$$

$\mathcal{L}^D = \mathcal{L}_{\mathrm{adv}}^D$

where the adversarial losses employ least-squares GAN (LSGAN) criteria for stability.

Network details:

• Generators: Two-stage cascaded U-Nets, each with residual convolutional blocks and three output heads predicting the cyclic image, scale-map ($\alpha$), and shape-map ($\beta$). $1/\alpha$ and $\beta$ are output with ReLU for positivity; $1/\alpha$ is inverted for numerical stability.
• Discriminators: 70×70 PatchGANs, 4–5 convolutional layers down to a 30×30 patch map, operating on real/fake samples.

Optimization uses Adam with $\beta_1=0.9$, $\beta_2=0.99$, $2\times10^{-4}$ initial learning rate, cosine annealing schedule, and typical values $(\lambda_1, \lambda_2) = (10, 2)$ (Upadhyay et al., 2021, Upadhyay et al., 2021).

3. Uncertainty Quantification: Aleatoric and Epistemic

UGAC provides closed-form aleatoric uncertainty from GGD parameters: $$\sigma^2_{\text{alea},ij} = \alpha_{ij}^2\, \frac{\Gamma(3/\beta_{ij})}{\Gamma(1/\beta_{ij})}$$ For epistemic uncertainty, Monte Carlo dropout at inference yields

$$\sigma^2_{\text{epis},ij} = \frac{1}{T} \sum_{t=1}^T \left( \hat u_{ij}^{(t)} - \frac{1}{T} \sum_{s=1}^T \hat u_{ij}^{(s)} \right)^2$$

Total predictive variance at each pixel is the sum

$$\sigma^2_{ij} = \sigma^2_{\text{alea},ij} + \sigma^2_{\text{epis},ij}$$

High $\alpha$ or low $\beta$ naturally arise in pixels under perturbation or with high reconstruction error, so uncertainty maps are spatially calibrated (Upadhyay et al., 2021).

4. Robustness and Adaptive Outlier Handling

By adaptively learning the residual distribution per pixel, UGAC robustly suppresses outlier influence and avoids over-penalization of high-residual regions—an effect maximized when testing under distribution shift. For Gaussian, uniform, or impulse (salt–pepper) noise that was not present during training, UGAC demonstrates that cycle-consistency loss is locally weighted by the per-pixel uncertainty, so reconstructions remain coherent even under severe perturbation (Upadhyay et al., 2021, Upadhyay et al., 2021).

Compared to disGAN, gcGAN, CycleGAN, nCyGAN and other baselines, UGAC yields substantially lower amplitude mean square error (AMSE) and higher amplitude structural similarity (ASSIM) metrics when evaluated on tasks including Cityscapes, Maps, Facades, and IXI MRI. For example, on Cityscapes with Gaussian noise:

• UGAC: $\text{AMSE}=80.2 \pm 10.4$, $\text{ASSIM}=0.78\pm 0.09$
• nCyGAN: $\text{AMSE}=107.8\pm11.2$, $\text{ASSIM}=0.60\pm0.08$

Visual results confirm that UGAC prevents catastrophic blurring and artifact introduction under maximum noise levels (Upadhyay et al., 2021, Upadhyay et al., 2021).

5. Experimental Results and Ablative Analysis

UGAC has been benchmarked on standard datasets:

• Cityscapes: 2,975 train / 500 test
• Google Maps: 1,096 images
• CMP Facades: 400 images
• IXI MRI: 15,000 train / 5,000 val / 10,000 test

Perturbation types include four levels each of Gaussian, uniform, and impulse noise, with metrics including IoU, AMSE, ASSIM, PSNR, and SSIM. UGAC consistently outperforms baselines not only in unperturbed scenarios but—with greater separation—as noise severity increases. For instance, on IXI (medical), under OOD demographic splits (age/weight), UGAC yields $\text{SSIM}\approx0.83/0.85$, $\text{PSNR}\approx25.8/26.3$ dB, outperforming deterministic $\ell_1/\ell_2$ cycle-consistency by more than $6$ dB (Upadhyay et al., 2021).

Ablation studies:

• Fixed Gaussian ($\beta=2$), Laplace ($\beta=1$), and learned GGD all considered. Learned GGD consistently superior.
• Uncertainty maps correlate positively with actual residual ($r \approx 0.85$), indicating calibration.
• Addition of uncertainty branches increases model complexity but is crucial for robustness.

6. Implementation Details and Limitations

Typical implementation uses cascaded U-Net generators, 6 downsampling+upsampling ResConv blocks, skip connections, three output heads. PatchGAN discriminators use spectral normalization. Dropout ($p=0.5$) regularizes for epistemic uncertainty estimation. Training on a single GPU (2080 Ti), $1,000$ epochs requires approximately $20$ hours for Cityscapes, with batch size $2$.

Limitations:

• Predicting two additional parameter maps ($\alpha, \beta$) increases computational and memory overhead.
• Extreme $\beta\to0$ situations may require explicit regularization.
• GAN instability and sensitivity to loss weighting remain; proper hyperparameter selection is necessary (Upadhyay et al., 2021, Upadhyay et al., 2021).

7. Relationship to Uncertainty-aware Cycle-consistency in Adverse Domain Transfer

Related works such as AU-GAN introduce heteroscedastic uncertainty branches in CycleGAN decoders using Laplacian likelihoods with per-pixel predicted variance. In adverse weather translation tasks (rainy-night $\to$ day), UA-cyc loss generalizes the standard cycle-consistency by explicitly down-weighting regions of high uncertainty, e.g., rain, glare, or artifact regions. Experiments on BDD100K and Alderley datasets show FID and mIoU gains for UA-cyc compared to standard cycle-loss and ablations both with and without feature transfer networks (Kwak et al., 2021). This indicates the broad relevance of uncertainty-aware adaptive cycle consistency beyond standard CycleGAN translation and into imbalanced and challenging domains.

• "Robustness via Uncertainty-aware Cycle Consistency" (Upadhyay et al., 2021)
• "Uncertainty-aware Generalized Adaptive CycleGAN" (Upadhyay et al., 2021)
• "Adverse Weather Image Translation with Asymmetric and Uncertainty-aware GAN" (Kwak et al., 2021)