WISL: Weighted Image Space Loss

• Weighted Image Space Loss (WISL) is a design pattern that applies explicit weights to reconstruction residuals, ensuring image-space fidelity while emphasizing clinically or perceptually significant features.
• It adapts weights according to factors such as diffusion time, perceptual salience, and lesion size, with variants ranging from pixel-wise and voxel-wise schemes to trainable and frequency-domain approaches.
• Empirical results demonstrate that WISL improves key metrics like SSIM and PSNR, though it introduces trade-offs in computational overhead and sensitivity to weight scheduling.

Searching arXiv for the specific term and directly related papers. arXiv search query: "Weighted Image Space Loss" WISL diffusion image space loss ControlNet latent diffusion MRI PET Weighted Image Space Loss (WISL) denotes a family of supervision terms in which image-space reconstruction residuals are multiplied by explicit weights before aggregation. In the most general sense represented in recent arXiv literature, WISL is not a single canonical formula but a design pattern: the loss remains anchored to image-space fidelity while modulating the contribution of pixels, voxels, frequencies, or diffusion time steps according to task-specific structure, perceptual salience, lesion size, or denoising stage (Sargood et al., 2 Aug 2025, Guo et al., 6 Jun 2026, Mellatshahi et al., 2023, Shirokikh et al., 2020, Czolbe et al., 2020). In diffusion-based MRI-to-PET synthesis, the term is used explicitly for a time-weighted image-space supervision term added to latent diffusion training (Sargood et al., 2 Aug 2025). In other contexts, closely related constructions appear under different names, including nonlinear weighted photometric losses for Gaussian Splatting, trainable pixel-wise loss weights for super-resolution, inverse-volume voxel weighting for segmentation, and perceptually weighted Fourier-domain losses for generative models (Guo et al., 6 Jun 2026, Mellatshahi et al., 2023, Shirokikh et al., 2020, Czolbe et al., 2020).

1. Definition and conceptual scope

A weighted image-space loss has the generic form

$$\mathcal{L}=\sum w \cdot \ell(\text{prediction}, \text{target}),$$

with the weighting applied directly in image space or in a transformed image-domain representation. In the LEGS formulation, a generic weighted image-space loss over pixels $p\in\Omega$ is written as

$$\mathcal{L}_{\text{WISL}}=\sum_{p\in\Omega} w_p\,\ell\!\big(I^{\text{render}}_p,I^{\text{gt}}_p\big),$$

where $w_p$ is a pixel-wise weight and $\ell$ is a base photometric loss (Guo et al., 6 Jun 2026). In super-resolution, the same principle appears as

$$L=\sum_{i=1}^{N} w_i\cdot \ell\big(f_\theta(y)_i,x_i\big),$$

with trainable pixel-wise weights $w_i\in[0,1]$ (Mellatshahi et al., 2023). In 3D medical segmentation, the same logic is extended to voxels, with each voxel assigned a component-dependent weight before insertion into Dice, Cross-Entropy, Focal, or Asymmetric Similarity losses (Shirokikh et al., 2020).

The unifying property is that WISL preserves an image-space target while abandoning uniform treatment of all locations or coefficients. The weighting signal may be fixed, learned, or analytically derived. It may depend on diffusion time $t$, structural operators such as the Laplacian, perceptual judges such as LPIPS, connected-component volumes, or perceptual thresholds in the frequency domain (Sargood et al., 2 Aug 2025, Guo et al., 6 Jun 2026, Mellatshahi et al., 2023, Shirokikh et al., 2020, Czolbe et al., 2020).

This suggests that WISL is best understood as a methodological category rather than a single loss. Its purpose is to retain the directness of image-space supervision while biasing optimization toward clinically meaningful, structurally salient, perceptually important, or otherwise underrepresented content.

2. Principal design axes

Across the cited literature, WISL variants differ along four recurrent axes: the support of the weights, the source of the weights, the base residual, and the training stage at which weighting is applied.

Axis	Variants appearing in the literature	Representative paper
Support	Time-step scalar, pixel-wise, voxel-wise, block-frequency coefficient	(Sargood et al., 2 Aug 2025, Guo et al., 6 Jun 2026, Shirokikh et al., 2020, Czolbe et al., 2020)
Weight source	Hand-designed schedule, structural prior, trainable network, inverse lesion volume, perceptual threshold	(Sargood et al., 2 Aug 2025, Guo et al., 6 Jun 2026, Mellatshahi et al., 2023, Shirokikh et al., 2020, Czolbe et al., 2020)
Base loss	$L_1$, MSE, BCE, Dice, Focal, ASL, Minkowski pooling in frequency space	(Sargood et al., 2 Aug 2025, Mellatshahi et al., 2023, Shirokikh et al., 2020, Czolbe et al., 2020)
Integration point	ControlNet training, Gaussian Splatting reconstruction, EM-style SR optimization, segmentation training, VAE reconstruction	(Sargood et al., 2 Aug 2025, Guo et al., 6 Jun 2026, Mellatshahi et al., 2023, Shirokikh et al., 2020, Czolbe et al., 2020)

In CoCoLIT, the weight is a scalar $\lambda_t$ tied to the diffusion denoising schedule, and the base residual is a 3D PET-space $p\in\Omega$0 difference computed after decoding an estimated clean latent (Sargood et al., 2 Aug 2025). In LEGS, the weights are per-pixel scalars derived from the normalized Laplacian response of the supervision image and are applied to the $p\in\Omega$1 photometric term while leaving the SSIM term unchanged (Guo et al., 6 Jun 2026). In trainable loss weights for super-resolution, the weights are sampled from a relaxed multivariate Bernoulli whose mean is predicted by a 4-layer CNN, under a fixed-sum constraint enforced by the FixedSum activation (Mellatshahi et al., 2023). In lesion segmentation, weights are inversely proportional to connected-component volume and normalized so that mean$p\in\Omega$2 (Shirokikh et al., 2020). In Watson-based generative training, the weighting is frequency- and content-dependent through perceptual thresholds $p\in\Omega$3, so that the effective weight is $p\in\Omega$4 in the Fourier domain (Czolbe et al., 2020).

A plausible implication is that WISL constructions can be classified by whether they emphasize when an error should matter, where it should matter, or how much it should matter relative to task structure.

3. Diffusion-aware WISL in latent MRI-to-PET synthesis

The most explicit use of the term “Weighted Image Space Loss” appears in “CoCoLIT: ControlNet-Conditioned Latent Image Translation for MRI to Amyloid PET Synthesis” (Sargood et al., 2 Aug 2025). There, CoCoLIT models the conditional distribution $p\in\Omega$5 in latent space, but the clinical target is the reconstructed PET volume $p\in\Omega$6. Purely latent-space objectives do not directly constrain the decoded PET output, and the proposed WISL introduces explicit image-space guidance during ControlNet training (Sargood et al., 2 Aug 2025).

The loss is defined as

$p\in\Omega$7

where $p\in\Omega$8 is the ground-truth PET volume, $p\in\Omega$9 is the PET VAE decoder, and $$\mathcal{L}_{\text{WISL}}=\sum_{p\in\Omega} w_p\,\ell\!\big(I^{\text{render}}_p,I^{\text{gt}}_p\big),$$0 is the estimate of the fully denoised latent at time-step $$\mathcal{L}_{\text{WISL}}=\sum_{p\in\Omega} w_p\,\ell\!\big(I^{\text{render}}_p,I^{\text{gt}}_p\big),$$1, computed via the DDPM reparameterization

$$\mathcal{L}_{\text{WISL}}=\sum_{p\in\Omega} w_p\,\ell\!\big(I^{\text{render}}_p,I^{\text{gt}}_p\big),$$2

with $$\mathcal{L}_{\text{WISL}}=\sum_{p\in\Omega} w_p\,\ell\!\big(I^{\text{render}}_p,I^{\text{gt}}_p\big),$$3 (Sargood et al., 2 Aug 2025).

Its distinctive feature is the time-dependent scalar schedule

$$\mathcal{L}_{\text{WISL}}=\sum_{p\in\Omega} w_p\,\ell\!\big(I^{\text{render}}_p,I^{\text{gt}}_p\big),$$4

This schedule is intended to respect the progressive nature of diffusion denoising: early steps recover coarse, low-frequency content, whereas later steps refine high-frequency details. The authors compare a constant image-space loss with $$\mathcal{L}_{\text{WISL}}=\sum_{p\in\Omega} w_p\,\ell\!\big(I^{\text{render}}_p,I^{\text{gt}}_p\big),$$5 against WISL’s time-dependent weighting and report that WISL yields better amyloid-related correlations and, in synergy with Latent Average Stabilization (LAS), the best overall performance (Sargood et al., 2 Aug 2025).

The ControlNet training loss is

$$\mathcal{L}_{\text{WISL}}=\sum_{p\in\Omega} w_p\,\ell\!\big(I^{\text{render}}_p,I^{\text{gt}}_p\big),$$6

with no additional scalar coefficient on $$\mathcal{L}_{\text{WISL}}=\sum_{p\in\Omega} w_p\,\ell\!\big(I^{\text{render}}_p,I^{\text{gt}}_p\big),$$7 (Sargood et al., 2 Aug 2025). Because $$\mathcal{L}_{\text{WISL}}=\sum_{p\in\Omega} w_p\,\ell\!\big(I^{\text{render}}_p,I^{\text{gt}}_p\big),$$8 depends on $$\mathcal{L}_{\text{WISL}}=\sum_{p\in\Omega} w_p\,\ell\!\big(I^{\text{render}}_p,I^{\text{gt}}_p\big),$$9, the PET decoder is explicitly fine-tuned during ControlNet training, while the backbone U-Net weights $w_p$0 are frozen (Sargood et al., 2 Aug 2025). WISL is used only in training Stage D, is computed on full decoded 3D PET reconstructions, and adds overhead because it requires a decoder forward pass for every sampled $w_p$1; the paper reports that CoCoLIT is implemented in MONAI and trained on an NVIDIA A100, and that the overhead is manageable in practice (Sargood et al., 2 Aug 2025).

The ablation study isolates the effect of temporal weighting. Relative to the constant-weight ISL variant, WISL improves A$w_p$2-related correlations at similar image metrics: for the configurations without LAS, CABC increases from $w_p$3 to $w_p$4, HABC from $w_p$5 to $w_p$6, while BA remains comparable at $w_p$7 versus $w_p$8 (Sargood et al., 2 Aug 2025). With LAS, the full CoCoLIT configuration reaches SSIM $w_p$9, PSNR $\ell$0, MSE $\ell$1, CABC $\ell$2, HABC $\ell$3, and BA $\ell$4 on the internal dataset (Sargood et al., 2 Aug 2025). The paper interprets this as evidence that time-weighted image-space guidance better captures clinically relevant SUVR patterns than a constant image-space penalty.

4. Pixel-wise and trainable forms

Outside diffusion, WISL most often appears as pixel-wise reweighting of a photometric or reconstruction residual. LEGS defines a Laplacian-enhanced weight map $\ell$5 and uses it to reweight the $\ell$6 photometric term,

$\ell$7

then combines this with an SSIM distance to form

$\ell$8

(Guo et al., 6 Jun 2026). The weights are derived from the supervision image through a Laplacian response

$\ell$9

followed by min–max normalization, a nonlinear response-to-weight mapping $$L=\sum_{i=1}^{N} w_i\cdot \ell\big(f_\theta(y)_i,x_i\big),$$0, and final weighting

$$L=\sum_{i=1}^{N} w_i\cdot \ell\big(f_\theta(y)_i,x_i\big),$$1

(Guo et al., 6 Jun 2026). LEGS does not alter the 3DGS rendering pipeline, Gaussian parameters, densification, or optimization variables; it is a loss-level modification, and the weight maps are precomputed from the ground-truth views (Guo et al., 6 Jun 2026). On Tanks-and-Temples and Mip-NeRF360, LEGS improves PSNR by up to $$L=\sum_{i=1}^{N} w_i\cdot \ell\big(f_\theta(y)_i,x_i\big),$$2 dB over 3DGS and up to $$L=\sum_{i=1}^{N} w_i\cdot \ell\big(f_\theta(y)_i,x_i\big),$$3 dB over EGGS, and the same second-order nonlinear weighting strategy improves FastGS and FasterGS by up to $$L=\sum_{i=1}^{N} w_i\cdot \ell\big(f_\theta(y)_i,x_i\big),$$4 dB (Guo et al., 6 Jun 2026).

“Trainable Loss Weights in Super-Resolution” presents a trainable pixel-wise weighting method that is explicitly a weighted image-space loss, even though the paper uses the term Trainable Loss Weights rather than WISL (Mellatshahi et al., 2023). The weighted pixel-wise loss is

$$L=\sum_{i=1}^{N} w_i\cdot \ell\big(f_\theta(y)_i,x_i\big),$$5

with $$L=\sum_{i=1}^{N} w_i\cdot \ell\big(f_\theta(y)_i,x_i\big),$$6, $$L=\sum_{i=1}^{N} w_i\cdot \ell\big(f_\theta(y)_i,x_i\big),$$7, and the weights sampled from a relaxed multivariate Bernoulli whose mean $$L=\sum_{i=1}^{N} w_i\cdot \ell\big(f_\theta(y)_i,x_i\big),$$8 is predicted by a weighting network (Mellatshahi et al., 2023). The FixedSum activation enforces the fixed-sum constraint while keeping the output components between zero and one: $$L=\sum_{i=1}^{N} w_i\cdot \ell\big(f_\theta(y)_i,x_i\big),$$9 The weighting network is optimized using an LPIPS-based criterion $w_i\in[0,1]$0, and the overall training uses an EM-style alternation between the super-resolution network and the weighting network (Mellatshahi et al., 2023).

Empirically, the paper reports that weighted image-space losses consistently improve LPIPS and often PSNR across RCAN, VDSR, EDSR, and HAT. For example, on RCAN $w_i\in[0,1]$1, Set5 improves from $w_i\in[0,1]$2 with $w_i\in[0,1]$3 to $w_i\in[0,1]$4 with $w_i\in[0,1]$5TLW, and Manga109 improves from $w_i\in[0,1]$6 to $w_i\in[0,1]$7 (Mellatshahi et al., 2023). The stated interpretation is that unweighted pixel losses treat all pixels equally, whereas learned weighting emphasizes visually important pixels under a controlled budget.

5. Voxel-wise and perceptual-frequency formulations

A distinct medical-imaging instantiation appears in “Universal Loss Reweighting to Balance Lesion Size Inequality in 3D Medical Image Segmentation” (Shirokikh et al., 2020). There, the weighting is defined over voxels rather than pixels and is inversely proportional to lesion volume. For a training patch with background $w_i\in[0,1]$8 and lesions $w_i\in[0,1]$9, with $t$0 and $t$1, the paper defines

$t$2

By construction, $t$3, so mean$t$4 (Shirokikh et al., 2020). These weights are then inserted into weighted Dice, BCE, Focal, and ASL losses. The paper’s central claim is that inverse weighting considerably increases detection quality while preserving delineation quality on a state-of-the-art level (Shirokikh et al., 2020). Representative hold-out results show lesion-wise Recall increases such as BCE on LUNA16 from $t$5 to $t$6, BCE on LiTS from $t$7 to $t$8, and Focal on metastases from $t$9 to $L_1$0 (Shirokikh et al., 2020).

A very different formulation appears in “A Loss Function for Generative Neural Networks Based on Watson’s Perceptual Model” (Czolbe et al., 2020). Here the weighted image-space loss is implemented in the frequency domain on $L_1$1 blocks. The amplitude term is

$L_1$2

where $L_1$3, and the effective weighting is $L_1$4, with $L_1$5 determined by Watson’s contrast sensitivity, luminance adaptation, and contrast masking (Czolbe et al., 2020). A phase penalty

$L_1$6

is added, and for color images the loss is aggregated across YCbCr channels with learned nonnegative weights $L_1$7 (Czolbe et al., 2020).

This paper explicitly frames the construction as a weighted image-space loss in the Fourier domain, with weights derived from perceptual thresholds rather than spatial salience or class imbalance (Czolbe et al., 2020). In a benchmark setting with batch size $L_1$8 and $L_1$9 images, Watson-DFT is reported as much lighter than LPIPS-VGG: for color, Watson-DFT uses $\lambda_t$0 s and $\lambda_t$1 MB, whereas LPIPS-VGG uses $\lambda_t$2 s and $\lambda_t$3 MB (Czolbe et al., 2020). The qualitative outcome reported is that VAEs trained with the new loss generate realistic, high-quality image samples with less blur than Euclidean distance and SSIM, and with less artifacts than deep neural network based losses (Czolbe et al., 2020).

6. Empirical themes and methodological trade-offs

Across these formulations, the empirical rationale for WISL is that uniform image-space penalties often fail to reflect the structure of the learning problem. In CoCoLIT, a constant image-space penalty may prematurely enforce fine detail generation early in the denoising process, potentially disrupting the learned trajectory; the proposed temporal weighting is intended to align supervision with progressive denoising (Sargood et al., 2 Aug 2025). In LEGS, standard photometric loss treats flat and structure-rich regions similarly, whereas Laplacian-based weights make the loss structure-aware while keeping the rendering pipeline unchanged (Guo et al., 6 Jun 2026). In super-resolution, unweighted pixel losses optimize PSNR or SSIM but may neglect perceptual aspects important to human vision, and trainable loss weights aim to preserve signal fidelity while guiding optimization toward perceptually meaningful pixels (Mellatshahi et al., 2023). In lesion segmentation, large lesions overshadow small ones unless per-lesion contributions are equalized through inverse-volume weights (Shirokikh et al., 2020). In the Watson-based loss, uniform Euclidean differences are replaced by perceptually normalized frequency differences (Czolbe et al., 2020).

Several trade-offs recur. First, weighting improves task alignment but usually adds computation. CoCoLIT requires decoder forward passes at each supervised diffusion step (Sargood et al., 2 Aug 2025). Trainable loss weights require a weighting network, LPIPS evaluations, and relaxed sampling (Mellatshahi et al., 2023). Frequency-domain Watson losses require blockwise FFTs and learned perceptual parameters, although the paper reports that this remains lightweight relative to LPIPS-VGG (Czolbe et al., 2020). Second, the choice of weighting function matters. LEGS reports that not all nonlinear mappings improve over the linear baseline; C3 performs best on Tanks-and-Temples, while C2 and C4 underperform the linear baseline (Guo et al., 6 Jun 2026). CoCoLIT notes sensitivity to the temporal schedule and mentions that other schedules such as cosine or exponential could be explored (Sargood et al., 2 Aug 2025). Super-resolution weighting depends on the LPIPS-based criterion and on the fixed-sum budget schedule

$\lambda_t$4

(Mellatshahi et al., 2023). Segmentation weighting may require weight caps or clipping to avoid extremely large weights for tiny lesions (Shirokikh et al., 2020).

A plausible implication is that WISL is most effective when the base task exhibits a known mismatch between uniform residual aggregation and the quantity that actually determines utility: clinical fidelity, perceptual similarity, structural sharpness, or small-object sensitivity.

7. Terminological ambiguity and related meanings of “WISL”

The acronym WISL is not unique to image losses. In sequence design and radar waveform design, WISL stands for Weighted Integrated Sidelobe Level, not Weighted Image Space Loss (Song et al., 2015, Eamaz et al., 2023). In “Sequence Design to Minimize the Weighted Integrated and Peak Sidelobe Levels,” WISL is defined for a unimodular sequence $\lambda_t$5 as

$\lambda_t$6

where $\lambda_t$7 is the aperiodic autocorrelation at lag $\lambda_t$8 (Song et al., 2015). In “Near-Field Low-WISL Unimodular Waveform Design for Terahertz Automotive Radar,” the same acronym denotes a correlation-based criterion over auto- and cross-correlation sidelobes of unimodular waveform sets (Eamaz et al., 2023).

This ambiguity is more than lexical. The image-loss usage concerns weighted residual aggregation in image reconstruction, generation, or segmentation, whereas the radar and sequence-design usage concerns weighted autocorrelation sidelobe energy. The two meanings share the abstract notion of weighting an error-like quantity, but they belong to different technical lineages and should not be conflated.

Within image-modeling papers themselves, terminology is also heterogeneous. CoCoLIT explicitly names its loss “Weighted Image Space Loss” (Sargood et al., 2 Aug 2025). LEGS describes a “nonlinear weighted loss” and does not use the acronym WISL, although its formulation is exactly a weighted image-space loss (Guo et al., 6 Jun 2026). The super-resolution paper uses “Trainable Loss Weights” rather than WISL (Mellatshahi et al., 2023). The segmentation paper describes a universal loss reweighting approach over voxels (Shirokikh et al., 2020). The Watson paper describes a perceptual weighted distance in frequency space (Czolbe et al., 2020). For encyclopedia purposes, the most precise usage is therefore to treat Weighted Image Space Loss as an umbrella concept for weighted image-domain supervision, while recognizing that the acronym WISL has an established and unrelated meaning in sidelobe optimization.

In summary, Weighted Image Space Loss refers to image-space supervision in which residuals are modulated by explicit weights before aggregation. Recent work instantiates this idea through diffusion-time schedules for latent denoising (Sargood et al., 2 Aug 2025), Laplacian-based pixel weighting for Gaussian Splatting (Guo et al., 6 Jun 2026), trainable perceptual pixel budgets for super-resolution (Mellatshahi et al., 2023), inverse-volume voxel equalization for lesion segmentation (Shirokikh et al., 2020), and Watson-derived perceptual thresholds in the Fourier domain (Czolbe et al., 2020). Across these formulations, the common objective is to preserve the directness of image-space losses while replacing uniform error aggregation with task-aligned weighting.