---
title: 'Pix2PixHD: Baseline for High-Res I2I Translation'
url: https://www.emergentmind.com/topics/pix2pixhd-baseline
type: topic
---

# Pix2PixHD: Baseline for High-Res I2I Translation

Pix2PixHD Baseline refers to the canonical architecture, loss functions, and training protocols described in Wang et al. [CVPR 2018] and widely adopted in subsequent image-to-image (I2I) translation research and challenge benchmarks. The model represents a key advance over the original pix2pix framework [1611.07004], scaling conditional GAN-based translation to high resolutions and providing robust baseline performance across a spectrum of domains—including satellite flood visualization [2010.08103], remote sensing [2207.00001], and binary-to-RGB translation from single-photon camera (SPC) data [2506.06890].

## 1. Architecture

### Generator

The baseline employs a two-stage, coarse-to-fine ResNet generator structure composed of:

- **Global generator (G₁):** 
    - Initial 7×7 convolution (64 filters, InstanceNorm, ReLU)
    - Two downsampling 3×3 conv layers (strided, 128→256 filters)
    - Nine ResNet blocks at ¼ input resolution
    - Two upsampling layers (nearest-neighbor upsample + 3×3 conv, 128→64 filters)
    - Final 7×7 conv to 3-channel RGB + Tanh nonlinearity

- **Local enhancer network (G₂, for 1024×1024+ inputs):**
    - 7×7 conv (32 filters), InstanceNorm, ReLU
    - Two downsampling conv layers (64→128 filters)
    - Three ResNet blocks
    - Two upsampling layers (64→32 filters), ending in 7×7 conv to 3-channel RGB + Tanh

For high-res synthesis, G₁’s output is bilinearly upsampled and fused via skip-connections into G₂. All non-output layers use InstanceNorm and ReLU. The model contains approximately 87 million parameters [2010.08103].

### Discriminators

Pix2PixHD baseline uses **three multi-scale PatchGAN discriminators** {D₁, D₂, D₃}, identical in structure but operating at full, ½, and ¼ resolution, respectively:

- Five 4×4 convolutions (stride=2): 64→128→256→512→1 channels
- LeakyReLU (slope=0.2) after all but last layer
- Spectral normalization on all convolutional layers (enforcing Lipschitz continuity, stabilizing GAN training)
- Input consists of the concatenation of the source and target (x, y) images

This multi-scale approach increases both stability and discriminative sensitivity to local and global artifacts.

## 2. Loss Functions

The Pix2PixHD baseline objective combines adversarial, feature-matching, and perceptual losses:

- **Adversarial loss (cGAN):**
  $$
  \mathcal{L}_{GAN}(G, D_i) = \mathbb{E}_{(x,y)}[\log D_i(x, y)] + \mathbb{E}_x[\log(1 - D_i(x, G(x)))]
  $$
  The total adversarial loss sums over all discriminators:
  $$
  \mathcal{L}_{GAN}(G, D) = \sum_{i=1}^3 \mathcal{L}_{GAN}(G, D_i)
  $$

- **Feature matching loss:**
  $$
  \mathcal{L}_{FM}(G, D) = \mathbb{E}_{(x, y)} \sum_{i=1}^3 \sum_{j=1}^{L_i} \frac{1}{N_{ij}} \| D_i^{(j)}(x, y) - D_i^{(j)}(x, G(x)) \|_1
  $$
  Here, $D_i^{(j)}(\cdot)$ is the feature map at the $j$-th layer of $D_i$.

- **Perceptual (VGG) loss:**
  $$
  \mathcal{L}_{VGG}(G) = \mathbb{E}_{(x, y)} \sum_{l=1}^L \frac{1}{M_l} \| \phi_l(y) - \phi_l(G(x)) \|_1
  $$
  $\phi_l(\cdot)$ is the activation of the $l$-th layer in a pretrained VGG-19.

The full generator objective is:
$$
G^* = \arg\min_G \left\{ \max_{D_1, D_2, D_3} \mathcal{L}_{GAN}(G, D) + \lambda_{FM} \mathcal{L}_{FM}(G, D) + \lambda_{VGG} \mathcal{L}_{VGG}(G) \right\}
$$
with $\lambda_{FM} = 10$, $\lambda_{VGG} = 10$ in typical setups [2010.08103]. Some applications may also include a pure $\ell_1$ pixel regression term (\(\lambda_{L1}\)), or replace the adversarial loss with its hinge variant [2506.06890].

## 3. Training Protocol

Key training parameters adopted across most reported baselines ([2010.08103], [2207.00001], [2506.06890]):

| Parameter                   | Typical Value                                       |
|-----------------------------|----------------------------------------------------|
| Batch size                  | 1 (per GPU)                                        |
| Optimizer                   | Adam ($\beta_1=0.5$, $\beta_2=0.999$)              |
| Initial learning rate       | $2 \times 10^{-4}$                                 |
| Learning rate schedule      | Linear decay to zero after 100 out of 200 epochs   |
| Number of epochs            | 200                                                |
| Normalization               | Instance normalization (all generator/discriminator layers) |
| Regularization              | Spectral normalization (discriminators, [miyato2018spectral]), no dropout in baseline |
| Data augmentation           | Random flip, rotation, crop, elastic deformation, hue/contrast shifts ([simard2003best], [2010.08103]) |

Dataset and resolution choices follow task-specific requirements, e.g., 1024×1024 for xBD flood visualization [2010.08103], 256×256 for satellite I2I [2207.00001], or inputs mirroring binary SPC data for novel-view synthesis [2506.06890].

## 4. Data Augmentation and Domain Adaptation

Standard augmentations—horizontal flipping, rotation, random cropping, elastic deformation—are routinely applied to increase effective sample diversity and support geometric robustness [2010.08103]. Input channel adaptation is performed as necessary (e.g., to accept multi-channel SAR bands [2207.00001] or 3-channel binary SPC frames [2506.06890]).

A plausible implication is that restricting augmentation (as in [2207.00001]) can lower the risk of distributional mismatch for geospatial or scientific data, whereas aggressive augmentation is useful for natural images or disaster scenarios.

No physics-based conditioning or auxiliary physical input channels are used in the unconditional Pix2PixHD “baseline” [2010.08103].

## 5. Quantitative Baseline Performance

Metrics are tailored to the application and dataset:

- **Physical-consistency (flood visualization):** Intersection-over-Union (IoU) between segmentation masks predicted from generated images and ground truth [2010.08103].
- **Photorealism:** Learned Perceptual Image Patch Similarity (LPIPS) [Zhang_2018]; PSNR and SSIM for regression or remote sensing tasks [2207.00001], [2506.06890].
- **Composite task metrics:** Flood Visualization Plausibility Score (FVPS), the harmonic mean of IoU and (1-LPIPS) [2010.08103].

Notable results:

| Domain / Metric          | Pix2PixHD Baseline Outcome               |
|--------------------------|------------------------------------------|
| Flood: IoU / LPIPS / FVPS| 0.226 / 0.293 / 0.275                    |
| MultiEarth (MAE, PSNR)   | 0.02258 MAE, 30.98 dB PSNR [2207.00001]  |
| SPC-to-RGB/PSNR/SSIM/LPIPS| 22.70 / 0.6843 / 0.4949 [2506.06890]    |

In all cases, Pix2PixHD sets a rigorous baseline for more specialized or physics-aware generative pipelines.

## 6. Relationship to Pix2Pix and Architectural Innovations

Pix2PixHD extends the original pix2pix baseline [1611.07004], which consisted of a U-Net generator and single-scale PatchGAN discriminator with a cGAN + L₁ loss, to larger scale and higher fidelity:

- **Key innovations:** 
    - Multi-scale (coarse-to-fine) generator capable of 1024×1024 or higher output
    - Three parallel, multi-scale PatchGAN discriminators
    - Feature matching loss for enhanced perceptual and statistical alignment
    - Perceptual (VGG) loss to reinforce high-level content similarity
    - InstanceNorm throughout, with removal of dropout

This modular architecture enables adaptation to diverse I2I translation regimes, from cross-modality satellite data [2207.00001] to pixel-level hallucination from binary input [2506.06890], preserving local realism and global structure under adversarial training.

## 7. Limitations and Comparisons

Across application domains, the Pix2PixHD baseline typically yields competitive realism and accuracy but is often outperformed by architectures leveraging spatially-adaptive normalization (SPADE), explicit physical constraints, or advanced ensemble strategies [2207.00001], [2010.08103]. Omitting specific loss terms (e.g., feature-matching) or architectural components systematically worsens performance, evidenced by PSNR drops and poorer perceptual scores [2506.06890].

A plausible implication is that while Pix2PixHD remains a robust and widely recognized baseline, domain adaptation, loss engineering, and data conditioning remain essential for pushing task-specific state of the art. Its rigorous structure and open implementation cement its role as the reference foundation for comparative evaluation in contemporary I2I research.

Source: https://www.emergentmind.com/topics/pix2pixhd-baseline