Frequency Distribution Loss in Image Transformation

• Frequency Distribution Loss is a spectral-domain objective that compares amplitude and phase distributions using Sliced Wasserstein Distance to overcome misalignment.
• FDL applies the Discrete Fourier Transform on feature maps (e.g., from VGG19), separating amplitude and phase to capture both global structure and local details.
• Empirical results demonstrate that FDL improves metrics such as PSNR, SSIM, and LPIPS in tasks like super-resolution, image enhancement, and style transfer.

Searching arXiv for the named topic and closely related frequency-domain loss papers to ground the article in current arXiv records. Frequency Distribution Loss (FDL) denotes a class of objectives that constrain learning through frequency-domain representations rather than solely through spatial-domain pointwise errors. In the most explicit arXiv formulation under that name, FDL is a misalignment-robust loss for image transformation: image features are transformed by the Discrete Fourier Transform (DFT), amplitude and phase are separated, and their distributions are compared with Sliced Wasserstein Distance (SWD) (Ni et al., 2024). Closely related work uses adjacent but non-identical ideas, including cross-entropy over local DCT-coefficient classes for JPEG restoration (Yoo et al., 2018), frequency-domain weighted reconstruction for super-resolution (Sims, 2020), low-frequency regularization of an embedded residual in steganography (Chen et al., 2023), and guided high-pass supervision for image restoration (Benjdira et al., 2023). This suggests that “Frequency Distribution Loss” is best treated as a family resemblance term whose precise meaning depends on what quantity is distributed, how frequency is represented, and which distance is minimized.

1. Definition and scope

In image transformation, FDL is defined as a distribution distance in the frequency domain, designed for paired settings in which the supervision may be geometrically misaligned. The central formulation compares the distributions of frequency-domain feature components rather than aligned pixels or aligned feature vectors. The predicted image $U$ and target image $V$ are first mapped into a perceptual feature space $\Phi$, typically VGG19, then transformed by the DFT, after which amplitude and phase are processed separately (Ni et al., 2024).

This usage is narrower than the broader set of frequency-aware losses. Some methods operate in the frequency domain but do not compare distributions; others compare distributions but not in the spectral sense. The distinction is important because the phrase “frequency distribution” is overloaded across arXiv literatures. In dialogue generation, for example, frequency refers to token frequencies in the corpus and the loss is a frequency-aware reweighting of cross-entropy rather than a spectral objective (Jiang et al., 2019). In differentially private table release, “frequency distribution” refers to sanitized multi-dimensional histograms and “loss” refers to disclosure risk from homogeneity attack rather than a training criterion (Liu et al., 2021). In actuarial risk, “frequency distribution” refers to claim counts that compound with severity to form an aggregate loss distribution (Witkovsky et al., 2017).

The image-transformation meaning is therefore specific: FDL is a spectral-distribution loss intended to retain robustness to spatial misalignment while constraining global structure more effectively than spatial-domain distribution matching (Ni et al., 2024).

2. Canonical formulation in image transformation

The defining formula of FDL is

$\mathcal{L}_{\text{FDL}\left(U, V \right) = \text{SW}\left(\mathcal{A}_{\Phi(U)},\mathcal{A}_{\Phi(V)} \right) + \lambda\cdot \text{SW}\left(\mathcal{P}_{\Phi(U)},\mathcal{P}_{\Phi(V)} \right),$

where $U$ is the predicted image, $V$ is the target image, $\Phi$ is a feature extractor, $\mathcal{A}_{\Phi(\cdot)}$ is the amplitude of the DFT of the feature map, $\mathcal{P}_{\Phi(\cdot)}$ is the corresponding phase, $\text{SW}(\cdot,\cdot)$ is Sliced Wasserstein Distance, and $V$0 balances phase relative to amplitude (Ni et al., 2024).

The implementation reported for image enhancement and super-resolution uses VGG19 as $V$1 and computes FDL on the $V$2, $V$3, $V$4, $V$5, and $V$6 layers (Ni et al., 2024). The paper does not write an explicit summation formula over layers, but it states that these layers are all used. The reported task-dependent settings are $V$7 for super-resolution and style transfer and $V$8 for image enhancement; an ablation also tests $V$9 on DPED, with $\Phi$0 best (Ni et al., 2024).

The same paper gives a style-transfer specialization. The style term is defined as

$\Phi$1

while the content term is

$\Phi$2

with $\Phi$3 the generated image, $\Phi$4 the style image, and $\Phi$5 the content image (Ni et al., 2024).

Several technical limits are explicit. The paper does not specify an explicit SWD formula in the main body, does not detail whether spectra are centered or log-scaled, and does not state any frequency masks or bandwise weighting beyond the scalar $\Phi$6 (Ni et al., 2024). Accordingly, FDL is defined operationally by its choice of representation and distance, not by a closed-form transport construction written out end to end.

3. Mechanism: amplitude, phase, and misalignment robustness

FDL is motivated by the failure of alignment-sensitive losses under even small spatial shifts. The paper contrasts two failure modes. First, pixel-wise and element-wise losses such as $\Phi$7, $\Phi$8, LPIPS, and cosine distance assume corresponding spatial positions, so geometric mismatch can induce large penalties and lead to blur or regression-to-the-mean behavior. Second, spatial-domain distribution losses such as CTX and PDL are more shift-tolerant, but the paper argues that they can ignore too much positional structure and permit disordered textures or structural corruption (Ni et al., 2024).

Its proposed remedy is to compute the distribution distance in the frequency domain, where global information is more explicit. The 1D toy formulation preceding the full loss is

$\Phi$9

with $\mathcal{L}_{\text{FDL}\left(U, V \right) = \text{SW}\left(\mathcal{A}_{\Phi(U)},\mathcal{A}_{\Phi(V)} \right) + \lambda\cdot \text{SW}\left(\mathcal{P}_{\Phi(U)},\mathcal{P}_{\Phi(V)} \right),$0 the Wasserstein Distance and $\mathcal{L}_{\text{FDL}\left(U, V \right) = \text{SW}\left(\mathcal{A}_{\Phi(U)},\mathcal{A}_{\Phi(V)} \right) + \lambda\cdot \text{SW}\left(\mathcal{P}_{\Phi(U)},\mathcal{P}_{\Phi(V)} \right),$1, $\mathcal{L}_{\text{FDL}\left(U, V \right) = \text{SW}\left(\mathcal{A}_{\Phi(U)},\mathcal{A}_{\Phi(V)} \right) + \lambda\cdot \text{SW}\left(\mathcal{P}_{\Phi(U)},\mathcal{P}_{\Phi(V)} \right),$2 the amplitude and phase spectra (Ni et al., 2024). The full image-space FDL replaces WD by SWD and applies the idea to feature maps.

The separation of amplitude and phase is not merely algebraic. Through a feature-swapping analysis using inverse DFT and a decoder, the paper interprets amplitude as carrying illumination, color, and texture-like appearance, while phase carries object shapes, edges, and structural arrangement (Ni et al., 2024). That interpretation motivates the two-term structure: amplitude matching constrains appearance statistics, phase matching constrains structure.

The robustness claim is empirical rather than theorem-driven. The paper reports a 1D toy experiment in which frequency-domain Wasserstein training remains robust to random shifts while giving better structural predictions than spatial-domain Wasserstein, and it shows a shift-response plot in which FDL changes much less than MSE or LPIPS when the reference image is shifted (Ni et al., 2024). This suggests that FDL is best viewed as a shift-tolerant structural distribution loss rather than a generic spectral regularizer.

4. Relation to adjacent frequency-aware objectives

A closely related but distinct formulation appears in JPEG restoration, where the problem is treated as estimating a discrete conditional distribution over local DCT coefficients. The image is converted to luminance $\mathcal{L}_{\text{FDL}\left(U, V \right) = \text{SW}\left(\mathcal{A}_{\Phi(U)},\mathcal{A}_{\Phi(V)} \right) + \lambda\cdot \text{SW}\left(\mathcal{P}_{\Phi(U)},\mathcal{P}_{\Phi(V)} \right),$3, a Laplacian image is computed, the image is divided into $\mathcal{L}_{\text{FDL}\left(U, V \right) = \text{SW}\left(\mathcal{A}_{\Phi(U)},\mathcal{A}_{\Phi(V)} \right) + \lambda\cdot \text{SW}\left(\mathcal{P}_{\Phi(U)},\mathcal{P}_{\Phi(V)} \right),$4 patches, DCT is applied to each patch, and the resulting $\mathcal{L}_{\text{FDL}\left(U, V \right) = \text{SW}\left(\mathcal{A}_{\Phi(U)},\mathcal{A}_{\Phi(V)} \right) + \lambda\cdot \text{SW}\left(\mathcal{P}_{\Phi(U)},\mathcal{P}_{\Phi(V)} \right),$5 coefficients are quantized into $\mathcal{L}_{\text{FDL}\left(U, V \right) = \text{SW}\left(\mathcal{A}_{\Phi(U)},\mathcal{A}_{\Phi(V)} \right) + \lambda\cdot \text{SW}\left(\mathcal{P}_{\Phi(U)},\mathcal{P}_{\Phi(V)} \right),$6 classes with equal numbers of training samples per bin. The network predicts a categorical distribution for each frequency channel and patch location, conceptually minimizing

$\mathcal{L}_{\text{FDL}\left(U, V \right) = \text{SW}\left(\mathcal{A}_{\Phi(U)},\mathcal{A}_{\Phi(V)} \right) + \lambda\cdot \text{SW}\left(\mathcal{P}_{\Phi(U)},\mathcal{P}_{\Phi(V)} \right),$7

and operationally uses a stagewise cross-entropy loss

$\mathcal{L}_{\text{FDL}\left(U, V \right) = \text{SW}\left(\mathcal{A}_{\Phi(U)},\mathcal{A}_{\Phi(V)} \right) + \lambda\cdot \text{SW}\left(\mathcal{P}_{\Phi(U)},\mathcal{P}_{\Phi(V)} \right),$8

This is distribution learning in the frequency domain, but it is classification over quantized local DCT coefficients rather than SWD over amplitude and phase distributions (Yoo et al., 2018).

By contrast, Frequency Domain-based Perceptual Loss (FDPL) is explicitly described as not being “a frequency distribution matching loss in the usual statistical sense.” It uses blockwise $\mathcal{L}_{\text{FDL}\left(U, V \right) = \text{SW}\left(\mathcal{A}_{\Phi(U)},\mathcal{A}_{\Phi(V)} \right) + \lambda\cdot \text{SW}\left(\mathcal{P}_{\Phi(U)},\mathcal{P}_{\Phi(V)} \right),$9 DCT on the luminance channel and compares aligned DCT coefficients of output and target with hand-crafted weighting derived from the JPEG luminance quantization table and a mean relative difference matrix:

$U$0

It is therefore a weighted per-frequency reconstruction loss, not a distribution-distance loss (Sims, 2020).

Low-frequency Image Deep Steganography (LIDS) uses yet another mechanism. A secret image $U$1 is mapped to an additive residual $U$2, low-pass filtered to $U$3, and inserted into the cover image by direct addition $U$4. The frequency loss is

$U$5

with

$U$6

This loss acts on the embedded residual rather than between prediction and target images, and it encourages low-frequency concentration rather than distributional matching between two image representations (Chen et al., 2023).

Guided Frequency Loss (GFL) is also frequency-aware but not distribution-based. Its global form is

$U$7

combining a Charbonnier component, a depth-1 Laplacian Pyramid component, and a Gradual Frequency component based on a progressively expanded high-pass filter (Benjdira et al., 2023). The paper presents GFL as a balanced spatial-plus-frequency supervision objective. A plausible implication is that GFL and FDL address a similar imbalance between easy low frequencies and hard high frequencies, but they do so through different mathematical mechanisms: one through frequency-guided reconstruction, the other through frequency-domain distribution distance.

5. Applications and empirical behavior

FDL is evaluated on image enhancement, real-world super-resolution, synthetic strongly misaligned super-resolution, and style transfer (Ni et al., 2024).

Setting	Model	Reported result
DPED image enhancement	NAFNet	FDL best on PSNR $U$8, SSIM $U$9, LPIPS $V$0, DISTS $V$1, FID $V$2
Real-world SR	SwinIR	FDL best on PSNR $V$3, SSIM $V$4, LPIPS $V$5, DISTS $V$6, FID $V$7
Synthetic misaligned DIV2K, Urban100	NLSN	FDL PSNR $V$8 vs CTX $V$9; SSIM $\Phi$0 vs $\Phi$1
Style transfer	Optimization-based	FDL reported to transfer structural information better than Gatys et al. and CTX

On DPED with NAFNet, FDL is reported as best on all metrics in the table: PSNR $\Phi$2, LPIPS $\Phi$3, DISTS $\Phi$4, SSIM $\Phi$5, and FID $\Phi$6. On SwinIR for enhancement, FDL is strongest overall with LPIPS $\Phi$7, DISTS $\Phi$8, SSIM $\Phi$9, and FID $\mathcal{A}_{\Phi(\cdot)}$0, while its PSNR $\mathcal{A}_{\Phi(\cdot)}$1 is slightly below LPIPS at $\mathcal{A}_{\Phi(\cdot)}$2 (Ni et al., 2024).

For merged real-world super-resolution data from RealSR and City100, FDL again dominates most metrics. With NAFNet it gives PSNR $\mathcal{A}_{\Phi(\cdot)}$3, DISTS $\mathcal{A}_{\Phi(\cdot)}$4, SSIM $\mathcal{A}_{\Phi(\cdot)}$5, and FID $\mathcal{A}_{\Phi(\cdot)}$6, while CTX is slightly better on LPIPS at $\mathcal{A}_{\Phi(\cdot)}$7 versus FDL $\mathcal{A}_{\Phi(\cdot)}$8. With SwinIR, FDL is best on every listed metric, including PSNR $\mathcal{A}_{\Phi(\cdot)}$9, LPIPS $\mathcal{P}_{\Phi(\cdot)}$0, DISTS $\mathcal{P}_{\Phi(\cdot)}$1, SSIM $\mathcal{P}_{\Phi(\cdot)}$2, and FID $\mathcal{P}_{\Phi(\cdot)}$3 (Ni et al., 2024).

The synthetic strongly misaligned DIV2K experiment is the clearest stress test of misalignment robustness. With NLSN, FDL improves Urban100 PSNR from CTX $\mathcal{P}_{\Phi(\cdot)}$4 to $\mathcal{P}_{\Phi(\cdot)}$5, LPIPS from $\mathcal{P}_{\Phi(\cdot)}$6 to $\mathcal{P}_{\Phi(\cdot)}$7, DISTS from $\mathcal{P}_{\Phi(\cdot)}$8 to $\mathcal{P}_{\Phi(\cdot)}$9, SSIM from $\text{SW}(\cdot,\cdot)$0 to $\text{SW}(\cdot,\cdot)$1, and FID from $\text{SW}(\cdot,\cdot)$2 to $\text{SW}(\cdot,\cdot)$3. On Set5, FDL increases PSNR from CTX $\text{SW}(\cdot,\cdot)$4 to $\text{SW}(\cdot,\cdot)$5 while also improving FID from $\text{SW}(\cdot,\cdot)$6 to $\text{SW}(\cdot,\cdot)$7 (Ni et al., 2024).

Ablations further specify what matters. On DPED with NAFNet, a spatial-domain SWD baseline,

$\text{SW}(\cdot,\cdot)$8

underperforms FDL on PSNR $\text{SW}(\cdot,\cdot)$9 vs $V$00, LPIPS $V$01 vs $V$02, DISTS $V$03 vs $V$04, SSIM $V$05 vs $V$06, and FID $V$07 vs $V$08. VGG19 is the best feature extractor among ResNet, EffNet, VGG19, and direct-pixel “None,” and the best phase weight for DPED is $V$09 (Ni et al., 2024).

These results support a fairly specific interpretation. FDL is not merely robust because it ignores alignment; it appears effective because it ignores alignment in a representation that still retains global structure. That is the paper’s central empirical claim.

6. Broader meanings and common distinctions

A common source of confusion is that not every frequency-aware loss is a frequency-distribution loss in the same mathematical sense. FDL compares distributions of DFT-derived amplitude and phase components with SWD (Ni et al., 2024). FDPL compares aligned DCT coefficients with fixed weights (Sims, 2020). LIDS compares a residual with its own low-pass filtered version to suppress high-frequency energy (Chen et al., 2023). GFL progressively expands high-pass supervision over training rather than matching a target spectral distribution (Benjdira et al., 2023). JPEG restoration by frequency-coefficient classification learns discrete conditional coefficient distributions, but those are class distributions over local DCT coefficients rather than SWD distances between amplitude and phase fields (Yoo et al., 2018).

The same terminological drift appears outside imaging. In dialogue generation, Frequency-Aware Cross-Entropy (FACE) addresses the fact that standard cross-entropy allocates more optimization pressure to frequent tokens. Its defining objective,

$V$10

is a token-frequency-aware reweighting of cross-entropy, not a spectral loss (Jiang et al., 2019). In differentially private histogram release, “frequency distribution loss” concerns disclosure risk under sanitization, with closed-form relationships between $V$11, $V$12, and homogeneity-attack risk rather than any neural training criterion (Liu et al., 2021). In compound-loss actuarial modeling, the “frequency distribution” is the claim-count law that enters the aggregate-loss characteristic function,

$V$13

and heavy-tailed loss frequencies can be modeled as mixtures of Negative Binomial and Poisson counts (Witkovsky et al., 2017, Dai et al., 2022).

This broader record suggests two editorial distinctions. First, “frequency” may mean spectral frequency, token occurrence frequency, claim count frequency, or histogram cell frequency. Second, “loss” may mean a trainable objective, a realized error distribution, or a privacy/disclosure loss. Within image transformation, however, Frequency Distribution Loss now has a relatively precise meaning: a misalignment-robust objective that computes a distribution distance between frequency-domain feature components, with amplitude and phase treated separately and optimized jointly (Ni et al., 2024).