---
title: Laplacian Pyramid GANs (LAPGAN)
url: https://www.emergentmind.com/topics/laplacian-pyramid-gans-lapgan
type: topic
---

# Laplacian Pyramid GANs (LAPGAN)

Laplacian Pyramid Generative Adversarial Networks (LAPGAN) model the synthesis of natural images as a hierarchical process operating across multiple scales. Originating from the work of Denton et al. (2015), LAPGAN exploits the Laplacian pyramid decomposition to break down image generation into conditional adversarial refinement stages, where each stage is responsible for synthesizing band-pass image components conditioned on coarser scale approximations. This coarse-to-fine structured approach advances over single-scale GANs by hierarchically capturing global structure, semantic content, and fine-grained texture, producing images with increased fidelity and perceptual realism [1506.05751].

## 1. Laplacian Pyramid Representation and Motivation

LAPGAN is premised on the observation that natural images exhibit strong cross-scale correlations: low-frequency (coarse) bands encode global geometric structure, while higher-frequency bands correspond to edges and texture. Formally, let $I_0 = I$ denote a color image of size $J \times J$. Define a downward operator $d(\cdot)$ for smoothing and decimation by two, yielding a Gaussian pyramid $\{I_0, I_1, ..., I_K\}$, where $I_{k+1} = d(I_k)$ recursively until $I_K$ is reduced to a small size (e.g., $8 \times 8$).

The Laplacian pyramid coefficients $\{h_0, ..., h_K\}$ are computed as
$$
h_k = I_k - u(I_{k+1}),\quad \text{for}\ k=0, ..., K-1;\qquad h_K = I_K,
$$
where $u(\cdot)$ is an upsampling operator that smooths then doubles the spatial size. Each $h_k$ is a band-pass image encoding detail at spatial frequencies between $J/2^{k+1}$ and $J/2^k$. The full image can be recursively reconstructed by
$$
I_k = u(I_{k+1}) + h_k,\quad \text{from}\ k=K\ \text{down to}\ 0.
$$
This cascaded representation allows LAPGAN to focus the synthesis task at each scale, simplifying learning and improving synthesis quality by capturing detail layer-wise [1506.05751].

## 2. Hierarchical GAN Architecture

At each scale $k$ of the Laplacian pyramid, LAPGAN employs a pair of neural networks, $(G_k, D_k)$, constituting a conditional GAN operating at that level. The generator $G_k$ synthesizes the band-pass image $\tilde{h}_k$ given a noise vector $z_k$ and a low-pass conditioning image $l_k \equiv u(I_{k+1})$. The discriminator $D_k$ receives either the true $h_k$ or synthesized $\tilde{h}_k$—each concatenated with the same $l_k$—and outputs a probability that the input is real. 

Network architectures are scale-dependent:
- At the coarsest scale ($k = K$), $G_K$ and $D_K$ are fully connected: e.g., $G_K$ maps a latent code $z_K$ through two hidden layers to an output image, and $D_K$ mirrors this structure with hidden units and sigmoid output.
- At finer scales, $G_k$ is a 3-layer convolutional network (e.g., 5×5 filters, increasing channels, batch normalization and ReLU activations). The noise $z_k$ is projected or tiled and concatenated as an additional channel, enabling stochastic detail.
- $D_k$ is a 2-layer convolutional network ending in a sigmoid, with analogous structure but fewer layers than the generators.

For higher-resolution targets (e.g., LSUN scenes at $64\times64$), generators and discriminators are correspondingly deeper convolutional stacks with larger filter sizes and feature maps [1506.05751].

## 3. Training Objectives and Procedure

LAPGAN applies the original GAN minimax game at each pyramid level. For $k < K$ (not the coarsest), the conditional adversarial objective is
$$
\min_{G_k}\max_{D_k}\ \mathbb{E}_{h_k, l_k \sim p_\text{data}} \left[ \log D_k(h_k, l_k) \right]
+ \mathbb{E}_{z_k \sim p_z, l_k \sim p_\text{data}} \left[ \log(1 - D_k(G_k(z_k,l_k), l_k)) \right].
$$
At $k=K$, the loss is the standard canonical GAN objective applied to the small image $I_K$ without conditioning. Networks at each pyramid level are trained independently using alternating stochastic gradient descent, and their model selection utilizes Parzen-window log-likelihood on validation splits. For data-limited domains (e.g., CIFAR10), data augmentation such as random cropping is employed to mitigate overfitting [1506.05751].

For class-conditional image synthesis, a class vector $c$ is appended to each $G_k$ and $D_k$ via a linear projection reshaped as a spatial map, enabling control over generated categories.

## 4. Sampling and Image Synthesis

Novel sample generation in LAPGAN proceeds via a coarse-to-fine reconstruction analogous to the Laplacian pyramid decoding process. For noise vectors $\{z_K, ..., z_0\}$:
- Initialize the coarsest image $\hat{I}_K = G_K(z_K)$.
- For $k = K-1$ downto $0$:
  1. Upsample the current image: $l_k = u(\hat{I}_{k+1})$.
  2. Generate the synthetic band-pass image: $\hat{h}_k = G_k(z_k, l_k)$.
  3. Aggregate to next finer level: $\hat{I}_k = l_k + \hat{h}_k$.
The process continues recursively until the finest resolution $\hat{I}_0$ is constructed [1506.05751].

An equivalent rephrasing in terms of residual (band-pass) synthesis is utilized in subsequent works such as MelanoGANs [1804.04338], with variants employing only the coarsest-scale latent code or adapting the upsampling operation (e.g., bilinear, deconvolution, or learned upsampling).

## 5. Quantitative and Qualitative Evaluation

LAPGAN achieves significant improvements in both quantitative log-likelihood and perceptual quality over single-scale GANs. On CIFAR10:
- Parzen-window log-likelihood: standard GAN $\approx -3617 \pm 353$; LAPGAN $\approx -1799 \pm 826$.
- Human “fooling” rate (percentage of times synthetic samples are labeled as real by evaluators): baseline GAN $< 10\%$, LAPGAN (unconditional) $\approx 35\%$, LAPGAN (class-conditional) $\approx 40\%$, real images $> 90\%$. Visual inspection reveals that LAPGAN generates samples with coherent object structure, sharp edges, and detail across scales, far exceeding single-GAN baselines. For higher-resolution datasets (e.g., LSUN 64x64), LAPGAN synthesizes structured scene images (e.g., church fronts, bedrooms) with realistic large-scale and fine-scale features [1506.05751].

The comparative study in MelanoGANs [1804.04338] supports these findings at higher resolutions (256×256). Key observations include:
- LAPGAN generates diverse, detailed samples but can exhibit high-frequency residual artifacts.
- Direct comparison of histogram metrics (JS divergence and Earth-Mover's Distance) with DCGAN and DDGAN (a modified LAPGAN): LAPGAN shows greater visual diversity but less accurate color histograms than DCGAN.
- In medical image synthesis (skin lesions), LAPGAN-derived synthetic augmentations can improve classifier validation accuracy over using only real data (LAPGAN samples: val acc 74.0%; baseline: 71.6%).

## 6. Variants, Extensions, and Practical Considerations

MelanoGANs [1804.04338] introduce a set of architectural and training modifications to LAPGAN, motivated by application to high-resolution and data-scarce domains. Notable changes include:
- Single-source noise: Only the coarsest scale generator receives explicit noise, while higher-level generators operate deterministically.
- Image-based discrimination: Discriminators at finer levels are tasked with classifying full images, rather than residual bands.
- Residual-deconvolution blocks: Generators for higher pyramid levels are structured as shallow residual networks (ResDeconv) applying learned corrections atop upsampled lower-resolution images.
- Learned upsampling: Replacement of fixed (e.g., bilinear) upsampling by learned deconvolution layers is explored, with measured artifact tradeoffs.
- End-to-end training: Generators and discriminators across all scales are trained jointly rather than independently, facilitating stability during training.

These modifications provide advantages in training stability, speed, and sample quality under application constraints, though may introduce additional artifacts or reduce diversity depending on configuration. Comparative evaluations show that DCGAN best matches color histograms, LAPGAN produces the most diversity and texture, and DDGAN (upsampling variant) yields a favorable balance between artifact suppression and sample variety at high resolution [1804.04338].

## 7. Impact and Applications

LAPGAN and its variants have driven advances in image synthesis by structurally aligning generative modeling with the intrinsic multiscale statistics of natural images, yielding substantial improvements in both visual realism and sample diversity relative to monolithic GAN architectures. Its hierarchical residual structure and independent per-scale modeling simplify generation at each stage, enabling synthesis of high-resolution images on limited data.

In practical settings, LAPGAN-generated images have been successfully applied to data augmentation tasks, such as compensating for class imbalance in medical imaging (melanoma lesion datasets), where synthetic samples bolster classifier accuracy in low-data regimes [1804.04338]. The methodology underlies subsequent multi-scale approaches in generative modeling, influencing research in conditional image synthesis, texture transfer, and even more recent hierarchical diffusion models.

A plausible implication is that further research could explore adaptive scale selection, multivariate hierarchical conditioning, or integration with alternate generative frameworks to enhance fidelity or interpretable control across scales.

Source: https://www.emergentmind.com/topics/laplacian-pyramid-gans-lapgan