DepthGAN: Unsupervised Stereo Depth Estimation

• The paper introduces an unsupervised approach using adversarial training to predict disparity maps with cycle-consistency constraints.
• The cycled generative architecture employs dual encoder-decoder CNNs with a shared ResNet-50 backbone to synthesize and validate stereo views.
• The method eliminates the need for expensive depth annotations by exploiting calibrated stereo pairs and geometric reconstruction techniques.

DepthGAN, also referred to in code as “unsup-stereo-DepthGAN”, is an unsupervised stereo depth estimation method that formulates disparity prediction as an adversarial image-generation problem within a calibrated stereo setting. Instead of regressing ground-truth depth labels, it predicts a correspondence field between left and right views, uses that field to synthesize one view from the other, and trains a pair of cycled generative networks with adversarial and consistency constraints. In the original formulation, the method is designed around stereo pairs for both training and inference, and it was introduced as “the first use of adversarial learning for unsupervised stereo image synthesis to improve depth estimation” (Pilzer et al., 2018).

1. Problem setting and geometric formulation

DepthGAN addresses dense depth estimation from a calibrated stereo camera with a left image $I_l$ and a right image $I_r$, assumed to be rectified so that corresponding points lie on the same scanline. The model predicts a disparity map $d$, where each pixel encodes the horizontal offset between corresponding pixels in the two views. Once disparity is available, depth is recovered through the stereo relation

$D = \frac{b_d \cdot f_l}{d},$

where $b_d$ is the camera baseline and $f_l$ is the focal length (Pilzer et al., 2018).

The method is explicitly unsupervised in the sense used by stereo self-supervision: it never uses ground-truth depth during training and instead relies on stereo pairs and camera geometry. This design targets a central limitation of supervised depth regression, namely the cost and difficulty of obtaining high-quality dense depth annotations through LiDAR, structured light, or dense multiview reconstruction. In that respect, DepthGAN belongs to the family of reconstruction-based stereo learning methods, but it departs from purely photometric formulations by placing disparity learning inside an adversarial synthesis framework.

A recurrent misconception is to treat DepthGAN as a monocular-depth method because it evaluates depth-like outputs and includes a “Half-Cycle Mono” baseline. The main model is not monocular at test time. Its intended operating regime is stereo: it consumes both left and right views, predicts disparities in both directions, and combines them into a final disparity estimate before converting to depth.

2. Cycled generative architecture

The architecture consists of two generator networks, two discriminators, and a cycle structure that links left-to-right and right-to-left synthesis (Pilzer et al., 2018).

The first generator, $G_l$, realizes the left-to-right half-cycle. It contains two sub-networks with the same encoder-decoder architecture. $G_l^{(l)}$ takes $I_l$ and predicts a left-to-right disparity $d_r^{(l)}$, while $I_r$0 takes $I_r$1 and predicts a second left-to-right disparity $I_r$2. These two disparity maps are fused by concatenation followed by a $I_r$3 convolution,

$I_r$4

and the fused disparity is then used to synthesize the right view through a differentiable warping operator with a bilinear sampler. The second generator, $I_r$5, mirrors this design in the reverse direction and reconstructs the left view from the synthesized right image.

Both generators use encoder-decoder CNNs with a ResNet-50 backbone in the encoder, 5 deconvolution layers with ReLU in the decoder, each upsampling by $I_r$6, and skip connections from encoder to decoder to fuse high-resolution appearance with deep features. A shared-encoder variant, in which the encoders of $I_r$7 and $I_r$8 share weights, performs best in