Normalized Laplacian Pyramid Distance

• Normalized Laplacian Pyramid Distance is a perceptual metric that simulates early human vision by combining photoreceptor compression, Laplacian pyramid decomposition, and divisive normalization.
• It accurately compares images across different dynamic ranges (HDR and LDR) and shows high correlation with human judgments, outperforming traditional measures like MSE and SSIM.
• Its fully differentiable implementation integrates into deep learning frameworks, optimizing tone mapping, image rendering, and GANs for enhanced perceptual fidelity.

The Normalized Laplacian Pyramid Distance (NLPD) is a full-reference perceptual image similarity metric designed for cross-dynamic-range image comparison and optimization. Its mathematical construction emulates major transformations of the early human visual system—photoreceptor nonlinearity, center-surround (Laplacian) decomposition, and divisive normalization—yielding a multiscale, gain-controlled representation in which perceptual distortions are more uniformly measurable across scales, spatial locations, and contexts. NLPD is distinguished by its ability to compare images with differing dynamic ranges (e.g., HDR/LDR) and its high correlation with human judgments of image quality. The metric is widely adopted for training and evaluation in tone mapping, image rendering, and deep learning frameworks where perceptual fidelity is paramount (Laparra et al., 2017, Le et al., 2021, Cao et al., 2022, Hepburn et al., 2019).

1. Physiological and Psychophysical Motivation

NLPD arises from the need to surpass the limitations of pixel-wise measures such as MSE, PSNR, and SSIM, which fail to capture perceptually salient differences when images are of different dynamic ranges. The metric is grounded in a three-stage physiological model of early vision:

• Static Photoreceptor Nonlinearity: The initial power-law compression (usually gamma ≈ 1/2.6) models cone photoreceptor response to real-world luminances.
• Center–Surround Multiscale Decomposition: Laplacian pyramid filtering simulates retina/LGN organization, yielding bandpass residuals at multiple scales.
• Divisive Normalization (Gain Control): Each band is normalized by a pooled estimate of local activity, emulating cortical contrast gain control and masking.

Each stage reflects empirical understanding of neural coding and psychophysical sensitivity, resulting in a representation highly aligned with human mean opinion scores on diverse natural and tone-mapped image sets (Laparra et al., 2017, Le et al., 2021).

2. Mathematical Definition and Construction

Let $S$ denote a calibrated reference image (often HDR) and $I$ a test or transformed image (often LDR). The transform from $S$ to its normalized Laplacian pyramid $\{Y^{(i)}\}_{i=1}^M$ involves:

• Stage 1: Photoreceptor Compression

$$S^{(1)}(x,y) = [S(x,y)]^\gamma,\quad \gamma\approx 1/2.6$$

• Stage 2: Laplacian Pyramid For levels $i = 1, \ldots, M-1$:

$$X^{(i+1)} = D[L\,X^{(i)}],\quad Z^{(i)} = X^{(i)} - U[L\,X^{(i+1)}]$$

The coarsest level: $Z^{(M)} = X^{(M)}$. Here, $L$ is a separable low-pass filter (binomial or Gaussian), $D$ downsampling by 2, $U$ upsampling with associated filtering.

• Stage 3: Divisive Normalization For each band $i$:

$$Y^{(i)}(x,y) = \frac{Z^{(i)}(x,y)}{[P^{(i)} * |Z^{(i)}|](x,y) + C_0^{(i)}}$$

$P^{(i)}$ is a small, local pooling kernel; $C_0^{(i)}$ is a stabilizing constant.

• Distance Calculation With normalized bands $\{Y^{(i)}\}$ for $S$ and $\{\widetilde Y^{(i)}\}$ for $I$,

$$\ell_{\rm NLPD}(S,I) = \left[ \frac{1}{M}\sum_{i=1}^M \left( \frac{1}{n^{(i)}} \sum_{j=1}^{n^{(i)}} |Y^{(i)}_j - \widetilde Y^{(i)}_j|^\alpha \right)^{\beta/\alpha} \right]^{1/\beta}$$

Typical choices: $\alpha \approx 2.0$, $\beta \approx 0.6$ (tuned to maximize correlation with human MOS), $M$ is the number of pyramid levels (Laparra et al., 2017, Cao et al., 2022, Le et al., 2021).

3. Algorithmic Description and Implementation

Efficient computation proceeds via in-place convolutions, separable filtering, and up/down-sampling. For practical configurations:

• Filters: Binomial $[0.05, 0.25, 0.4, 0.25, 0.05]$, Gaussian alternatives also common.
• Normalization kernel: Separably applied, typically $5\times5$ support.
• Stabilization constants: $C_0^{(i)} = 0.17$ (bandpass), $C_0^{(M)} = 4.86$ (coarsest band).
• All operations (convolutions, power-law, down/upsample, normalization) are differentiable almost everywhere, hence suitable for back-propagation in deep networks.

A condensed pseudocode implementation appears in several sources:

S1 = S ** gamma X_S[1] = S1 for i in range(1, M): LXi = convolve(X_S[i], L) X_S[i+1] = downsample(LXi) Up = upsample(convolve(X_S[i+1], L)) Z_S[i] = X_S[i] - Up Z_S[M] = X_S[M] for i in range(1, M+1): pool = convolve(abs(Z_S[i]), P[i]) Y_S[i] = Z_S[i] / (pool + C0[i]) sumBands = 0 for i in range(1, M+1): diff = abs(Y_S[i] - Y_I[i]) ** alpha meanDiff = mean(diff) sumBands += meanDiff ** (beta/alpha) dist = (sumBands / M) ** (1/beta)

(Cao et al., 2022, Laparra et al., 2017, Hepburn et al., 2019)

4. Parameters, Calibration, and Band Weighting

Parameter choices are empirically optimized for perceptual agreement:

Parameter	Value or Range	Purpose
$\gamma$	$\approx 1/2.6$	Photoreceptor compression
$M$ (pyramid levels)	$4$–$6$	Multi-scale coverage
$L$, $P^{(i)}$	Binomial, Gaussian	Pyramid and normalization filtering
$C_0^{(i)}$	$0.17$–$4.86$	Divisive normalization stabilization
$\alpha$, $\beta$	$2.0$, $0.6$	Minkowski exponents (perceptual pooling)
Band weights	Usually uniform	May be tuned per application

Parameters are set based on physiological models and fitted using large human-rated databases (e.g., TID2008) to maximize MOS correlation. Minor variants employ band-dependent weightings to approximate contrast sensitivity functions (Laparra et al., 2017, Cao et al., 2022).

5. Applications in Perceptual Optimization and Deep Learning

NLPD is used as a perceptual loss or optimization objective in several domains:

• Tone Mapping and HDR Rendering: Deep neural networks for tone mapping are optimized end-to-end using NLPD loss, resulting in LDR outputs with enhanced fidelity to HDR ground truth in both global and local contrast (Cao et al., 2022, Le et al., 2021).
• Constrained Display Rendering: Direct optimization over display constraints (black/white levels, mean luminance, halftoning) minimizes NLPD between displayed and original scene, producing visually preferred solutions without manual tuning (Laparra et al., 2017).
• Conditional GANs: NLPD is integrated as a perceptual regularizer replacing or augmenting $L_1$ loss, yielding generated images with sharper edges and realistic shading. Empirical studies reveal improvements in segmentation accuracy, no-reference image quality scores, and user preference rates (Hepburn et al., 2019).

NLPD's fully differentiable structure ensures compatibility with backpropagation and GPU acceleration for both training and evaluation (Le et al., 2021, Hepburn et al., 2019).

6. Perceptual Validation and Quantitative Benchmarks

Extensive studies on standard databases demonstrate NLPD's high empirical alignment with human perceptual judgments:

• Correlation: Pearson’s $r \approx 0.94$ on TID2008 cross-dynamic-range subsets, consistently outperforming SSIM, MS-SSIM, VIF, and PSNR by 5–10% in MOS correlation (Laparra et al., 2017, Le et al., 2021).
• Detection Thresholds: Near-ideal Weber-law scaling for normalized coefficients, perceptual masking effects for local contrast.
• Subjective Experiments: Iterative minimization ("NLPD-Opt") ranks near the top in 2AFC judgments on HDR scenes and tone-mapped outputs (Cao et al., 2022).
• GAN Applications: Semantic segmentation accuracy and no-reference metrics (BRISQUE, NIQE) improve under NLPD regularization compared to $L_1$ (Hepburn et al., 2019).

7. Strengths, Limitations, and Extension Prospects

Strengths:

• Biologically plausible construction enables direct alignment with early vision sensitivity and masking.
• Validated on diverse distortion types, including cross-dynamic-range and masked images.
• Fully differentiable, enabling use as a loss in optimization and deep learning.

Limitations:

• Original formulation is grayscale-only and does not handle color opponency natively.
• Computational cost is higher than block-based metrics due to local normalization per band.
• May under-predict visibility of large structured or geometric distortions due to locality.

Practical Extensions:

• Recommend precomputing or learning filter weights for efficient implementations.
• For real-time applications, surrogate feed-forward networks may approximate NLPD.
• Calibration of $\varepsilon$ and band weights may be advisable if metric is applied to new distortion types or color channels (Laparra et al., 2017, Cao et al., 2022).

NLPD continues to be actively employed and adapted in perceptual image quality assessment, tone mapping, image synthesis, and generative modeling frameworks for both classical and deep learning paradigms.