Depth-Aware CycleGAN: A Geometric Perspective

• Depth-Aware CycleGAN is a family of models that integrate explicit depth maps and geometric priors to enforce cycle-consistency on structural features rather than purely on appearance.
• These models adapt conventional CycleGAN architectures by fusing depth cues via mechanisms like disparity prediction, depth-conditioned loss modulation, and geometric masking.
• Empirical results across applications such as stereo depth estimation and underwater enhancement demonstrate improved structural fidelity and translation performance.

Depth-Aware CycleGAN denotes a family of cycle-consistent adversarial frameworks in which depth is treated as an explicit geometric variable rather than as a by-product of appearance translation. In the literature, this designation spans several distinct but related constructions: geometry-aware disparity prediction with stereo warping, depth-conditioned image enhancement, RGB-to-depth translation for completion, unpaired RGB-D domain adaptation, and anatomically constrained medical image translation. What unifies these systems is the replacement of uniform image-to-image translation by objectives or architectures that incorporate depth maps, disparity fields, geometric masks, structural priors, or depth-specific consistency terms into the CycleGAN paradigm (Pilzer et al., 2018, Ghosh, 2024, Wang et al., 2023, Mathew et al., 2020).

1. Scope of the term and representative formulations

The literature suggests that “Depth-Aware CycleGAN” is not a single canonical model, but a class of CycleGAN-derived systems in which depth enters the model either as an output, an input, a conditioning signal, or a domain-specific prior. Standard CycleGAN uses two generators and two discriminators with adversarial and cycle-consistency losses; depth-aware variants modify that template to reflect the geometry of the task rather than treating all pixels or modalities uniformly (Pilzer et al., 2018, Baruhov et al., 2020).

Representative usages reported in the literature include the following (Pilzer et al., 2018, Ghosh, 2024, Mathew et al., 2020, Wang et al., 2023, Baruhov et al., 2020, Kirch et al., 2022, Zhang et al., 2020, Zhang et al., 15 Sep 2025):

Paper	Task	Depth-aware mechanism
"Unsupervised Adversarial Depth Estimation using Cycled Generative Networks" (Pilzer et al., 2018)	Unsupervised stereo depth estimation	Disparity prediction, differentiable warping, disparity consistency
"Separated Attention: An Improved Cycle GAN Based Under Water Image Enhancement Method" (Ghosh, 2024)	Underwater image enhancement	Depth-conditioned foreground/background masking in the loss
"Augmenting Colonoscopy using Extended and Directional CycleGAN for Lossy Image Translation" (Mathew et al., 2020)	OC↔VC translation and depth inference	Extended cycle in VC/depth domain and Directional Discriminator
"RDFC-GAN: RGB-Depth Fusion CycleGAN for Indoor Depth Completion" (Wang et al., 2023)	Indoor depth completion	RGB→depth CycleGAN fused with Manhattan-guided regression
"Unsupervised Enhancement of Real-World Depth Images Using Tri-Cycle GAN" (Baruhov et al., 2020)	Unpaired depth enhancement	Masked depth losses and tri-cycle consistency
"VoloGAN: Adversarial Domain Adaptation for Synthetic Depth Data" (Kirch et al., 2022)	Synthetic-to-real RGB-D adaptation	Joint 4-channel RGB-D translation with depth-weighted losses
"Multi-task GANs for Semantic Segmentation and Depth Completion with Cycle Consistency" (Zhang et al., 2020)	Semantic translation and depth completion	Semantic-conditioned dense depth generation
"BREA-Depth: Bronchoscopy Realistic Airway-geometric Depth Estimation" (Zhang et al., 15 Sep 2025)	Bronchoscopic depth estimation	Joint RGB-depth translation with airway structure awareness

A central distinction from vanilla CycleGAN is that the cycle no longer operates solely on appearance. In stereo depth estimation, the cycle is mediated by rectified stereo geometry and bilinear warping rather than by direct appearance reconstruction (Pilzer et al., 2018). In underwater enhancement, “attention” is implemented purely at the loss level through depth-conditioned masking, with no explicit self-attention blocks (Ghosh, 2024). In medical translation, the cycle may be extended into a geometry-preserving domain such as VC or airway depth, so that structural information rather than texture is the invariant quantity (Mathew et al., 2020, Zhang et al., 15 Sep 2025).

2. Principal modes of depth awareness

One major mode is geometry-aware prediction, in which the generators do not produce stylized images but correspondence fields or depth maps that are constrained by image formation geometry. In the stereo model of Pilzer et al., the generators predict left-to-right and right-to-left disparity fields, and the cycle is closed through differentiable warping,

$$\hat I_r(x) = \mathcal{W}(I_l, d_{l\to r})(x), \qquad \hat I_l(x) = \mathcal{W}(I_r, d_{r\to l})(x),$$

with depth recovered from disparity through

$Z(x) = \frac{fB}{d(x)}.$

This makes the system “depth-aware” because cycle-consistency is enforced on geometry-aligned disparities rather than on unconstrained image translations (Pilzer et al., 2018).

A second mode is depth-conditioned modulation of the loss or features. In the underwater enhancement model, a monocular depth map $D \in [0,1]$ from Depth Anything is used to split an image into foreground and background components with $w(d)=d$ and $w_{\mathrm{bg}}(d)=1-d$. Foreground and background are optimized in parallel, with separate adversarial and cycle-consistency terms. The paper explicitly states that no transformer/self-attention or squeeze-and-excitation module is inserted; the “Separated Attention” mechanism is a training-time masking and weighting strategy rather than a feature-space attention block (Ghosh, 2024). In RDFC-GAN, depth-awareness instead enters through multi-stage W-AdaIN fusion, where a depth latent $z$ modulates RGB features and self-attention weights combine RGB and depth statistics, while Manhattan-normal constraints stabilize planar indoor structure (Wang et al., 2023).

A third mode is domain priors that privilege geometry over appearance. In XDCycleGAN for colonoscopy, the OC→VC mapping is lossy because VC does not admit patient-specific textures or specular highlights. The extended cycle consistency loss therefore compares geometry in the VC domain instead of forcing exact OC reconstruction, and the Directional Discriminator receives paired concatenations such as $\{OC, G_{vc}(OC)\}$ and $\{G_{oc}(VC), VC\}$ to discriminate the translation direction (Mathew et al., 2020). In the multi-task segmentation-depth framework, generated semantic images are concatenated with sparse depth and RGB to condition dense depth completion, and a semantic-guided smoothness loss aligns depth discontinuities with semantic boundaries (Zhang et al., 2020). In BREA-Depth, depth-awareness is anatomical: synthetic airway geometry is paired with depth, and an airway structure awareness loss enforces the prior that the lumen should be deeper than non-lumen regions (Zhang et al., 15 Sep 2025).

A fourth mode is depth-specific handling of missingness, sensor asymmetry, or modality coupling. Tri-Cycle GAN treats holes as informative rather than arbitrary, masks cycle and identity losses on the high-quality depth side, and introduces a tri-cycle that relaxes exact low-quality reconstruction under one-to-many degradation (Baruhov et al., 2020). VoloGAN processes early-fused RGB-D tensors end-to-end and uses per-channel cycle, identity, and SSIM weighting with $\lambda^d=3$ and $\lambda^{r,g,b}=1$ to reduce RGB-to-depth channel pollution (Kirch et al., 2022).

3. Objective design

The defining characteristic of depth-aware CycleGAN variants is not merely the presence of a depth branch, but the insertion of depth into the optimization objective.

For underwater enhancement, the final objective is depth-weighted and branch-specific:

$Z(x) = \frac{fB}{d(x)}.$0

with foreground and background masks derived from the depth map, $Z(x) = \frac{fB}{d(x)}.$1, foreground attention weight $Z(x) = \frac{fB}{d(x)}.$2, and background attention weight $Z(x) = \frac{fB}{d(x)}.$3. The method uses LSGAN adversarial losses and L1 cycle-consistency, and identity loss is not used (Ghosh, 2024).

For unsupervised stereo depth estimation, the depth-aware cycle is geometric:

$Z(x) = \frac{fB}{d(x)}.$4

with $Z(x) = \frac{fB}{d(x)}.$5, $Z(x) = \frac{fB}{d(x)}.$6, and $Z(x) = \frac{fB}{d(x)}.$7. Here $Z(x) = \frac{fB}{d(x)}.$8 is photometric L1 reconstruction, $Z(x) = \frac{fB}{d(x)}.$9 is the adversarial loss over reconstructed views, and $D \in [0,1]$0 enforces consistency between disparity maps after geometric alignment. Smoothness and explicit occlusion masks are not used (Pilzer et al., 2018).

For lossy medical translation, the central modification is the extended cycle:

$D \in [0,1]$1

This relieves the system from hiding OC textures or specular highlights inside synthetic VC images. The full XDCycleGAN objective combines the extended cycle, standard cycle in the VC→OC direction, directional losses in both directions, adversarial regularization of reconstructed OC, and an identity loss on VC, with $D \in [0,1]$2, $D \in [0,1]$3, and $D \in [0,1]$4 (Mathew et al., 2020).

For real-world depth enhancement under asymmetric domains, Tri-Cycle GAN adds

$D \in [0,1]$5

together with masked cycle and identity losses based on a validity mask $D \in [0,1]$6 if $D \in [0,1]$7, and a preserve loss for range stabilization. This formulation is explicitly motivated by the fact that $D \in [0,1]$8 is one-to-many, so exact low-quality reconstruction is unnecessarily strict (Baruhov et al., 2020).

For indoor depth completion, RDFC-GAN aggregates a local regression branch and a CycleGAN-style RGB→depth branch:

$D \in [0,1]$9

with $w(d)=d$0. The CycleGAN component itself includes $w(d)=d$1, $w(d)=d$2, $w(d)=d$3, $w(d)=d$4, and

$w(d)=d$5

while Manhattan-normal losses regularize plane orientation (Wang et al., 2023).

For anatomically constrained bronchoscopy, BREA-Depth introduces

$w(d)=d$6

and the total objective

$w(d)=d$7

with $w(d)=d$8, $w(d)=d$9, $w_{\mathrm{bg}}(d)=1-d$0, and $w_{\mathrm{bg}}(d)=1-d$1 (Zhang et al., 15 Sep 2025).

These objectives show that “depth-aware” generally means one of three things: the cycle is defined in a geometry-preserving space, the loss is modulated by depth, or additional priors make depth physically, semantically, or anatomically consistent. This suggests that the decisive innovation is often in the loss rather than in the discriminator-generator template.

4. Architectural patterns and training practice

Architecturally, depth-aware CycleGAN variants often retain familiar GAN backbones while changing the semantics of the channels or the loss interfaces. The underwater “Separated Attention” model keeps a U-Net generator with skip connections between mirrored layers and a Markovian PatchGAN that judges $w_{\mathrm{bg}}(d)=1-d$2 patches. Inputs are resized to $w_{\mathrm{bg}}(d)=1-d$3, the implementation is in PyTorch, the model is trained on 11K paired EUVP images with learning rate $w_{\mathrm{bg}}(d)=1-d$4, batch size $w_{\mathrm{bg}}(d)=1-d$5, and 100 epochs on a single NVIDIA GeForce RTX 3080 (Ghosh, 2024).

The stereo depth model uses two geometry-aware generators arranged in a cycle, each built around a ResNet-50 encoder and a decoder with five deconvolution layers and skip connections. A differentiable bilinear sampler implements warping, and PatchGAN discriminators operate on reconstructed views. Training is in TensorFlow on KITTI and Cityscapes, with Adam, initial learning rate $w_{\mathrm{bg}}(d)=1-d$6, batch size $w_{\mathrm{bg}}(d)=1-d$7, and a staged schedule culminating in 100K iterations of joint fine-tuning (Pilzer et al., 2018).

RDFC-GAN departs more strongly from the vanilla template. Its Manhattan-Constraint Network uses a PSPNet-guided normal module and a ResNet-18 encoder-decoder to regress a local dense depth map and confidence map, while the RGB-Depth Fusion CycleGAN branch uses another ResNet-18-based generator and PatchGAN discriminators. Fusion occurs through W-AdaIN at multiple intermediate stages. Training lasts 150 epochs; the MCN branch uses AdamW with initial learning rate $w_{\mathrm{bg}}(d)=1-d$8, while the CycleGAN branch uses Adam with initial learning rate $w_{\mathrm{bg}}(d)=1-d$9 (Wang et al., 2023).

VoloGAN adapts synthetic RGB-D to consumer-sensor RGB-D using full 4-channel tensors of size $z$0. The generators are six-level U-Nets with spectral normalization, instance normalization, reflection padding, spatial dropout, and a gated self-attention block at the $z$1 decoder stage. Its discriminator is not PatchGAN but a full-image, SIV-GAN-inspired architecture with low-level, content, and layout outputs. Training uses TPUv3 hardware, NADAM for generators, SGD for discriminators, warm-up plus cosine decay, and a global batch size of $z$2 (Kirch et al., 2022).

Medical variants are similarly heterogeneous. XDCycleGAN is trained on 10 paired VC/OC acquisitions rendered or cropped to $z$3, for 200 epochs, with spectral normalization applied to each discriminator layer (Mathew et al., 2020). BREA-Depth uses a U-Net Transformer-like encoder-decoder with multi-head outputs for RGB and depth, PatchGAN discriminators, 9,500 synthetic image-depth pairs, 55,000 real bronchoscopic frames, batch size $z$4, learning rate $z$5, and 30 epochs; inference is reported at 60 FPS (Zhang et al., 15 Sep 2025). The multi-task semantic-depth model downsamples Cityscapes images to $z$6, KITTI depth inputs to $z$7, and trains with a CycleGAN-style schedule in which the learning rate is $z$8 for the first 100 epochs and decays linearly to zero over the next 100 epochs (Zhang et al., 2020).

The recurrent architectural pattern is therefore conservative: U-Net, ResNet, PatchGAN, or closely related modules remain dominant. The distinctive element is usually the way geometry is injected, fused, or regularized.

5. Empirical behavior across application domains

Reported gains are task-specific rather than directly comparable, but they show that depth-aware modifications usually improve the target geometry or structure metric for the domain at hand.

In underwater enhancement, the depth-conditioned model was evaluated on 1K test images. Mean PSNR improved from 23.41 ± 2.41 for CycleGAN and 23.49 ± 2.92 for FUnIE-GAN to 23.79 ± 2.53 for SEP-Attn; SSIM improved from 0.729 ± 0.033 for CycleGAN to 0.741 ± 0.046; and UIQM improved from 3.03 ± 0.306 for CycleGAN to 3.17 ± 0.302. In a user study with 66 participants, SEP-Attn achieved 48% rank-1, 62% rank-2, and 83% rank-3 selections. The paper also reports that L1 loss contributes approximately 2.9% improvement and separated attention approximately 7.25%, for a net additional approximately 4.35% over L1 alone (Ghosh, 2024).

In unsupervised stereo depth estimation, the full cycle with adversarial training reduced KITTI errors relative to half-cycle baselines. On KITTI ablations without cropping, “Full-Cycle + D” obtained Abs Rel 0.198, Sq Rel 1.990, RMSE 6.655, RMSE log 0.292, and $z$9 of 0.721, while the shared-encoder version reached Abs Rel 0.190 and $\{OC, G_{vc}(OC)\}$0 of 0.751. In the state-of-the-art comparison on the Eigen split with Garg crop, the shared-encoder model at 50 m achieved Abs Rel 0.144, Sq Rel 1.007, RMSE 4.660, RMSE log 0.240, and $\{OC, G_{vc}(OC)\}$1 of 0.968, approaching the unsupervised method of Godard et al. (2017) (Pilzer et al., 2018).

In colonoscopy, the extended-cycle and directional formulation improves geometric consistency in the VC→OC→VC case from 7.74 ± 6.07 for CycleGAN to 6.84 ± 5.39 for XCycleGAN and 6.34 ± 3.73 for XDCycleGAN. When trained on VC depth maps, it achieved average SSIM 0.918 across 2000 frames and RMSE 31.25 ± 6.76 on a manually textured VC flythrough, compared with SSIM 0.637 and RMSE 92.67 ± 10.32 for Ma et al. The method is also reported to ignore specular highlights as false depth more effectively than Mahmood et al. (Mathew et al., 2020).

In indoor depth completion, RDFC-GAN reports state-of-the-art results on both NYU-Depth V2 and SUN RGB-D. On NYU-Depth V2 Setting A, it achieved RMSE 0.120, Rel 0.012, $\{OC, G_{vc}(OC)\}$2, $\{OC, G_{vc}(OC)\}$3, and $\{OC, G_{vc}(OC)\}$4, improving over RDF-GAN and GraphCSPN. On SUN RGB-D Setting A, it achieved RMSE 0.214, Rel 0.040, $\{OC, G_{vc}(OC)\}$5, $\{OC, G_{vc}(OC)\}$6, and $\{OC, G_{vc}(OC)\}$7. In point-cloud evaluation on NYU-Depth V2 Setting A, it reported Chamfer distance $\{OC, G_{vc}(OC)\}$8 m and F1 0.95 (Wang et al., 2023).

In unpaired depth enhancement, Tri-Cycle GAN improved synthetic-data PNCC from 0.668 for base CycleGAN to 0.879, and DROT PNCC from 0.213 for base CycleGAN to 0.633, slightly above the SDS baseline at 0.611. The paper attributes the gain to masked depth losses, larger generator capacity, and the tri-cycle term (Baruhov et al., 2020).

In multi-task semantic-depth completion, the best semantic-conditioned variant achieved KITTI RMSE 746.96, MAE 267.71, iRMSE 2.24, and iMAE 1.10, improving on ablations without the semantic-guided smoothness term. On Cityscapes semantic translation, the proposed model reported 0.623 per-pixel accuracy, 0.258 per-class accuracy, and 0.176 class IoU for image→label (Zhang et al., 2020).

In bronchoscopy, BREA-Depth reported DepthCon 97.27% and LocalAccu 62.36% on the ex vivo dataset, outperforming Depth Anything, Depth Anything V2, and EndoOmni-B/L. Ablations showed large drops without the CycleGAN component, to DepthCon 68.36 and LocalAccu 25.36, and notable drops without the airway loss, to 96.67 and 52.06. On the phantom dataset with median alignment, it reported Abs Rel 0.23 and RMSE 12.26 (Zhang et al., 15 Sep 2025).

6. Limitations, misconceptions, and research directions

A common misconception is that “depth-aware” necessarily implies explicit attention blocks or a physics-based restoration model. The underwater model explicitly rejects both interpretations: its “Separated Attention” is applied at the loss level, not through self-attention layers, and it does not use the underwater image formation model $\{OC, G_{vc}(OC)\}$9 with $\{G_{oc}(VC), VC\}$0 (Ghosh, 2024). Conversely, some models do use actual feature attention, such as W-AdaIN self-attention in RDFC-GAN or the gated self-attention block in VoloGAN (Wang et al., 2023, Kirch et al., 2022). The term therefore refers to any design in which depth changes the optimization or representation, not to a single architectural motif.

Another misconception is that vanilla cycle-consistency is sufficient for depth-sensitive translation. Several papers motivate additional constraints precisely because the standard $\{G_{oc}(VC), VC\}$1 assumption breaks under lossy or asymmetric mappings. XDCycleGAN introduces an extended cycle because OC→VC discards texture and specular information (Mathew et al., 2020). Tri-Cycle GAN introduces $\{G_{oc}(VC), VC\}$2 because high-quality-to-low-quality depth degradation is one-to-many (Baruhov et al., 2020). The stereo model replaces appearance-domain cycles with geometry-aligned disparity consistency (Pilzer et al., 2018), while the multi-task and bronchoscopy models add semantic or anatomical priors to regulate depth discontinuities and lumen structure (Zhang et al., 2020, Zhang et al., 15 Sep 2025).

The limitations are likewise domain-specific. The underwater model reports approximately 36% increased processing time, “zitter” artifacts at foreground/background transitions, and sensitivity to monocular depth errors under domain shifts or extreme turbidity (Ghosh, 2024). The stereo model assumes static scenes and rectified stereo pairs and does not use explicit occlusion handling or smoothness regularization, making reflective or textureless regions difficult (Pilzer et al., 2018). XDCycleGAN struggles with heavy occlusion, motion blur, fluid motion, instruments, and subtle protrusions that blend with wall texture (Mathew et al., 2020). RDFC-GAN may struggle on extreme transparency, specularity, or highly non-Manhattan scenes (Wang et al., 2023). VoloGAN reports occasional color wash-out, mild depth tilt, and segmentation-induced thinning of limbs (Kirch et al., 2022). The multi-task model still faces Cityscapes-to-KITTI domain gaps and slightly worse depth MAE than some baselines (Zhang et al., 2020). BREA-Depth remains sensitive to threshold-based lumen masks and lacks explicit centerline or topology-aware loss terms for narrow distal branches (Zhang et al., 15 Sep 2025).

The literature suggests several recurring directions for further work. These include explicit occlusion modeling and edge-aware smoothness in stereo depth (Pilzer et al., 2018), temporal constraints in endoscopy (Mathew et al., 2020), efficient formulations of stronger geometric priors beyond Manhattan assumptions (Wang et al., 2023), multi-view consistency and uncertainty estimation for depth enhancement (Baruhov et al., 2020), and more anatomy-specific structural constraints for bronchoscopic depth (Zhang et al., 15 Sep 2025). A plausible implication is that future depth-aware CycleGAN systems will be judged less by appearance realism alone and more by whether the cycle encodes the physically or anatomically correct invariant for the target domain.