Papers
Topics
Authors
Recent
Search
2000 character limit reached

Noise2Contrast: Noise-Guided Denoising

Updated 9 July 2026
  • Noise2Contrast is a research motif that uses noise as a supervisory signal, contrasting independent noise sources to recover image details and enhance restoration.
  • It spans methods from self-supervised tomographic denoising with multi-contrast fusion to low-light enhancement via noise self-regression, each exploiting unique noise structures.
  • The framework enables learning transferable noise representations while addressing modality-specific challenges such as registration issues and gray-world biases.

Noise2Contrast is a term with both narrow and broad usage in recent arXiv literature. In its narrowest and most explicit sense, it denotes a self-supervised tomographic denoising framework that exploits independently noisy multi-contrast acquisitions by stacking a denoiser with a domain-transfer operator and training against another contrast without clean targets (Wagner et al., 2022). In a broader, descriptive sense, the term is also used for methods that learn restoration or noise modeling from noise structure itself: low-light enhancement via noise self-regression in NoiSER (Zhang et al., 2022), contrastive noise embedding and single-reference noise transfer in NoiseTransfer (Lee et al., 2023), and, less formally, contrastive estimation of camera-specific noise parameters for realistic RAW noise synthesis (Zou et al., 2022). Across these usages, the recurring theme is that noise is not treated merely as corruption to be removed, but as a supervisory signal, an alignment cue, or an identifiable distributional object.

1. Terminology and scope

The term “Noise2Contrast” is not used uniformly. One paper adopts it as the official method name, whereas others use it as a paradigm label or as an interpretive shorthand for related contrastive or self-regressive ideas.

Context Paper Use of the term
Tomographic denoising "Noise2Contrast: Multi-Contrast Fusion Enables Self-Supervised Tomographic Image Denoising" (Wagner et al., 2022) Official method name
Low-light enhancement "Noise Self-Regression: A New Learning Paradigm to Enhance Low-Light Images Without Task-Related Data" (Zhang et al., 2022) Described as a Noise2Contrast paradigm
Noise generation "NoiseTransfer: Image Noise Generation with Contrastive Embeddings" (Lee et al., 2023) Connected to a Noise2Contrast paradigm
RAW noise estimation "Estimating Fine-Grained Noise Model via Contrastive Learning" (Zou et al., 2022) Not officially named Noise2Contrast

This variation matters because the underlying technical objects being “contrasted” differ substantially. In tomographic denoising, the contrast is literal imaging contrast across co-registered acquisitions. In NoiSER, the central mechanism is self-regression on pure Gaussian noise, with contrast referring to local differences between adjacent pixels. In NoiseTransfer and the fine-grained RAW noise model paper, contrastive learning is used to separate noise distributions or noise parameters in representation space (Wagner et al., 2022, Zhang et al., 2022, Lee et al., 2023, Zou et al., 2022).

A plausible implication is that “Noise2Contrast” functions less as a single algorithmic family than as a research motif: methods use noise-derived structure to recover image contrast, to exploit multi-contrast independence, or to infer transferable noise representations.

2. Noise self-regression and low-light image enhancement

In low-light image enhancement, NoiSER formulates a Noise2Contrast paradigm in which a compact convolutional neural network is trained only on pure Gaussian noise patches and then applied to real low-light inputs at inference (Zhang et al., 2022). The target task is LLIE, seeking a mapping fθf_\theta from a low-light image Ilow∈RH×W×3I_{\text{low}} \in \mathbb{R}^{H \times W \times 3} to a normal-light image Inorm∈RH×W×3I_{\text{norm}} \in \mathbb{R}^{H \times W \times 3} with corrected illumination, preserved structure, and natural color. Training uses no task-related data. Instead, each training sample is i.i.d. Gaussian noise,

In(u,v,c)∼N(0,σ2),I_n(u,v,c) \sim \mathcal{N}(0,\sigma^2),

and the network is optimized by self-regression,

arg⁡min⁡θEx{L(fθ(x),x)},\arg\min_\theta \mathbb{E}_x \{L(f_\theta(x),x)\},

with x=Inx=I_n and a loss composed of ℓ1\ell_1 reconstruction and TV regularization:

Ltotal=Lℓ1(fθ(In),In)+Ltv(fθ(In)).\mathcal{L}_{\text{total}}=\mathcal{L}_{\ell_1}(f_\theta(I_n),I_n)+\mathcal{L}_{tv}(f_\theta(I_n)).

The architecture is a lightweight self-regression model with approximately $1$K parameters, two non-linear activation layers, instance normalization, and no shortcuts or residual connections. The role of instance normalization is central. For each instance and channel,

yc=γc⋅xc−μcσc2+ϵ+βc,y_c=\gamma_c \cdot \frac{x_c-\mu_c}{\sqrt{\sigma_c^2+\epsilon}}+\beta_c,

with learnable Ilow∈RH×W×3I_{\text{low}} \in \mathbb{R}^{H \times W \times 3}0 and Ilow∈RH×W×3I_{\text{low}} \in \mathbb{R}^{H \times W \times 3}1. The reported interpretation is threefold: self-regression reconstructs local contrast between adjacent pixels, instance normalization remediates global illumination, and the zero-mean Gaussian training distribution biases outputs toward the gray-world hypothesis when the image is sufficiently large (Zhang et al., 2022).

The method’s reported behavior depends on variance choice and stopping time. Standard Gaussian training with Ilow∈RH×W×3I_{\text{low}} \in \mathbb{R}^{H \times W \times 3}2 is used in NoiSER-FC and NoiSER-ES, while NoiSER-Var3 uses Ilow∈RH×W×3I_{\text{low}} \in \mathbb{R}^{H \times W \times 3}3 for improved visual effect. Inputs and outputs are normalized to Ilow∈RH×W×3I_{\text{low}} \in \mathbb{R}^{H \times W \times 3}4, where Ilow∈RH×W×3I_{\text{low}} \in \mathbb{R}^{H \times W \times 3}5 corresponds to the mid-tone Ilow∈RH×W×3I_{\text{low}} \in \mathbb{R}^{H \times W \times 3}6 in Ilow∈RH×W×3I_{\text{low}} \in \mathbb{R}^{H \times W \times 3}7. Optimization uses Adam with Ilow∈RH×W×3I_{\text{low}} \in \mathbb{R}^{H \times W \times 3}8, Ilow∈RH×W×3I_{\text{low}} \in \mathbb{R}^{H \times W \times 3}9, learning rate Inorm∈RH×W×3I_{\text{norm}} \in \mathbb{R}^{H \times W \times 3}0, batch size Inorm∈RH×W×3I_{\text{norm}} \in \mathbb{R}^{H \times W \times 3}1, and Inorm∈RH×W×3I_{\text{norm}} \in \mathbb{R}^{H \times W \times 3}2 noise patches. Full convergence uses Inorm∈RH×W×3I_{\text{norm}} \in \mathbb{R}^{H \times W \times 3}3 iterations, whereas an early-stopping variant uses approximately Inorm∈RH×W×3I_{\text{norm}} \in \mathbb{R}^{H \times W \times 3}4 iterations to trade slight quantitative performance for better visual clarity (Zhang et al., 2022).

On LOL, the reported results are: NoiSER-FC with PSNR Inorm∈RH×W×3I_{\text{norm}} \in \mathbb{R}^{H \times W \times 3}5, SSIM Inorm∈RH×W×3I_{\text{norm}} \in \mathbb{R}^{H \times W \times 3}6 and NIQE Inorm∈RH×W×3I_{\text{norm}} \in \mathbb{R}^{H \times W \times 3}7; NoiSER-ES with PSNR Inorm∈RH×W×3I_{\text{norm}} \in \mathbb{R}^{H \times W \times 3}8, SSIM Inorm∈RH×W×3I_{\text{norm}} \in \mathbb{R}^{H \times W \times 3}9 and NIQE In(u,v,c)∼N(0,σ2),I_n(u,v,c) \sim \mathcal{N}(0,\sigma^2),0; and NoiSER-Var3 with PSNR In(u,v,c)∼N(0,σ2),I_n(u,v,c) \sim \mathcal{N}(0,\sigma^2),1, SSIM In(u,v,c)∼N(0,σ2),I_n(u,v,c) \sim \mathcal{N}(0,\sigma^2),2 and NIQE In(u,v,c)∼N(0,σ2),I_n(u,v,c) \sim \mathcal{N}(0,\sigma^2),3. Training and inference costs are also reported: NoiSER-FC requires approximately In(u,v,c)∼N(0,σ2),I_n(u,v,c) \sim \mathcal{N}(0,\sigma^2),4 min training time, NoiSER-ES approximately In(u,v,c)∼N(0,σ2),I_n(u,v,c) \sim \mathcal{N}(0,\sigma^2),5 min, inference is approximately In(u,v,c)∼N(0,σ2),I_n(u,v,c) \sim \mathcal{N}(0,\sigma^2),6 ms, and the model has approximately In(u,v,c)∼N(0,σ2),I_n(u,v,c) \sim \mathcal{N}(0,\sigma^2),7 parameters. Cross-dataset evaluation on LSRW reports Huawei SSIM In(u,v,c)∼N(0,σ2),I_n(u,v,c) \sim \mathcal{N}(0,\sigma^2),8 for NoiSER-FC and Nikon PSNR In(u,v,c)∼N(0,σ2),I_n(u,v,c) \sim \mathcal{N}(0,\sigma^2),9 for NoiSER-ES, with NoiSER-Var3 achieving Nikon SSIM arg⁡min⁡θEx{L(fθ(x),x)},\arg\min_\theta \mathbb{E}_x \{L(f_\theta(x),x)\},0. The same mapping is also reported to suppress overexposure on SICE, because the gray-world pull and instance-normalization rescaling move extreme brightness toward more moderate outputs (Zhang et al., 2022).

The principal limitations follow directly from the mechanism. Scenes that violate the gray-world hypothesis may be over-neutralized, very small images weaken the law-of-large-numbers argument on channel means, heavily sensor-noisy inputs may have noise emphasized rather than removed, and full convergence at arg⁡min⁡θEx{L(fθ(x),x)},\arg\min_\theta \mathbb{E}_x \{L(f_\theta(x),x)\},1 can introduce slightly gray veiling. The paper’s practical guidance is correspondingly conservative: retain instance normalization, keep the compact no-shortcut design, include TV regularization, and use early stopping or arg⁡min⁡θEx{L(fθ(x),x)},\arg\min_\theta \mathbb{E}_x \{L(f_\theta(x),x)\},2 when visual crispness is more important than maximal PSNR or SSIM (Zhang et al., 2022).

3. Multi-contrast fusion for self-supervised tomographic denoising

In its most specific usage, Noise2Contrast is a self-supervised denoising framework for tomographic imaging that exploits routine multi-contrast acquisitions such as MRI T1, T2, and FLAIR or dual-energy CT/X-ray microscopy (Wagner et al., 2022). The formal setting assumes images

arg⁡min⁡θEx{L(fθ(x),x)},\arg\min_\theta \mathbb{E}_x \{L(f_\theta(x),x)\},3

with zero-mean noise and independence across contrasts for arg⁡min⁡θEx{L(fθ(x),x)},\arg\min_\theta \mathbb{E}_x \{L(f_\theta(x),x)\},4:

arg⁡min⁡θEx{L(fθ(x),x)},\arg\min_\theta \mathbb{E}_x \{L(f_\theta(x),x)\},5

The framework introduces a denoiser arg⁡min⁡θEx{L(fθ(x),x)},\arg\min_\theta \mathbb{E}_x \{L(f_\theta(x),x)\},6 that maps a noisy image to a denoised image in the same contrast, and a domain-transfer operator arg⁡min⁡θEx{L(fθ(x),x)},\arg\min_\theta \mathbb{E}_x \{L(f_\theta(x),x)\},7 that maps clean content from contrast arg⁡min⁡θEx{L(fθ(x),x)},\arg\min_\theta \mathbb{E}_x \{L(f_\theta(x),x)\},8 to contrast arg⁡min⁡θEx{L(fθ(x),x)},\arg\min_\theta \mathbb{E}_x \{L(f_\theta(x),x)\},9. The self-supervised loss is

x=Inx=I_n0

with MSE as the loss function.

The key methodological point is separation of denoising from contrast transfer. Two denoising realizations are studied. The first uses three stacked trainable bilateral filter layers with learnable spatial and range kernel widths; by construction, these filters cannot perform contrast transfer and therefore preserve task separation. The second uses a network operator-based denoiser such as a U-Net, but this is trained sequentially with a frozen domain-transfer network to avoid the denoiser “cheating” by altering contrast rather than denoising (Wagner et al., 2022).

The domain-transfer network is a standard U-Net with about x=Inx=I_n1 million parameters. Training uses Adam with learning rate x=Inx=I_n2 and proceeds until self-supervised validation loss or training loss converges, depending on modality. The reported MRI pairings are T1x=Inx=I_n3T2, T2x=Inx=I_n4T1, and T2x=Inx=I_n5FLAIR; the reported XRM pairings are LEx=Inx=I_n6HE and HEx=Inx=I_n7LE. No masking or blind-spot mechanism is required, because supervision comes from an independently noisy target contrast rather than from pixel-wise independence assumptions (Wagner et al., 2022).

The reported results show that Noise2Contrast with bilateral filters outperforms the noisy baseline and the compared self-supervised alternatives. On brain MRI, T1x=Inx=I_n8T2 improves from baseline PSNR/SSIM x=Inx=I_n9 and ℓ1\ell_10 to ℓ1\ell_11 and ℓ1\ell_12; T2ℓ1\ell_13T1 improves from ℓ1\ell_14 and ℓ1\ell_15 to ℓ1\ell_16 and ℓ1\ell_17; T2ℓ1\ell_18FLAIR improves from ℓ1\ell_19 and Ltotal=Lℓ1(fθ(In),In)+Ltv(fθ(In)).\mathcal{L}_{\text{total}}=\mathcal{L}_{\ell_1}(f_\theta(I_n),I_n)+\mathcal{L}_{tv}(f_\theta(I_n)).0 to Ltotal=Lℓ1(fθ(In),In)+Ltv(fθ(In)).\mathcal{L}_{\text{total}}=\mathcal{L}_{\ell_1}(f_\theta(I_n),I_n)+\mathcal{L}_{tv}(f_\theta(I_n)).1 and Ltotal=Lℓ1(fθ(In),In)+Ltv(fθ(In)).\mathcal{L}_{\text{total}}=\mathcal{L}_{\ell_1}(f_\theta(I_n),I_n)+\mathcal{L}_{tv}(f_\theta(I_n)).2. The paper summarizes these as gains of Ltotal=Lℓ1(fθ(In),In)+Ltv(fθ(In)).\mathcal{L}_{\text{total}}=\mathcal{L}_{\ell_1}(f_\theta(I_n),I_n)+\mathcal{L}_{tv}(f_\theta(I_n)).3–Ltotal=Lℓ1(fθ(In),In)+Ltv(fθ(In)).\mathcal{L}_{\text{total}}=\mathcal{L}_{\ell_1}(f_\theta(I_n),I_n)+\mathcal{L}_{tv}(f_\theta(I_n)).4 in PSNR and Ltotal=Lℓ1(fθ(In),In)+Ltv(fθ(In)).\mathcal{L}_{\text{total}}=\mathcal{L}_{\ell_1}(f_\theta(I_n),I_n)+\mathcal{L}_{tv}(f_\theta(I_n)).5–Ltotal=Lℓ1(fθ(In),In)+Ltv(fθ(In)).\mathcal{L}_{\text{total}}=\mathcal{L}_{\ell_1}(f_\theta(I_n),I_n)+\mathcal{L}_{tv}(f_\theta(I_n)).6 in SSIM over the noisy baseline. On dual-energy XRM, LELtotal=Lℓ1(fθ(In),In)+Ltv(fθ(In)).\mathcal{L}_{\text{total}}=\mathcal{L}_{\ell_1}(f_\theta(I_n),I_n)+\mathcal{L}_{tv}(f_\theta(I_n)).7HE improves from Ltotal=Lℓ1(fθ(In),In)+Ltv(fθ(In)).\mathcal{L}_{\text{total}}=\mathcal{L}_{\ell_1}(f_\theta(I_n),I_n)+\mathcal{L}_{tv}(f_\theta(I_n)).8 and Ltotal=Lℓ1(fθ(In),In)+Ltv(fθ(In)).\mathcal{L}_{\text{total}}=\mathcal{L}_{\ell_1}(f_\theta(I_n),I_n)+\mathcal{L}_{tv}(f_\theta(I_n)).9 to $1$0 and $1$1, while HE$1$2LE improves from $1$3 and $1$4 to $1$5 and $1$6, corresponding to reported gains of $1$7–$1$8 in PSNR and $1$9–yc=γc⋅xc−μcσc2+ϵ+βc,y_c=\gamma_c \cdot \frac{x_c-\mu_c}{\sqrt{\sigma_c^2+\epsilon}}+\beta_c,0 in SSIM (Wagner et al., 2022).

The method’s assumptions are explicit. Accurate alignment between contrasts is important; imperfect registration pushes the transfer network to absorb geometric compensation. The fidelity of yc=γc⋅xc−μcσc2+ϵ+βc,y_c=\gamma_c \cdot \frac{x_c-\mu_c}{\sqrt{\sigma_c^2+\epsilon}}+\beta_c,1 matters, because an imperfect transfer operator can bias the gradients seen by the denoiser. The method also requires at least two contrasts per subject. Its advantage over blind-spot methods is that it does not rely on pixel-wise independent noise, and the XRM results are presented precisely in a regime where real correlated noise undermines that assumption (Wagner et al., 2022).

4. Contrastive noise embeddings and single-reference transfer

NoiseTransfer extends the broader Noise2Contrast idea from denoising to noise generation (Lee et al., 2023). Its objective is not to denoise directly, but to learn a single generative model that can synthesize noisy images with multiple different noise distributions and can transfer the noise characteristics of a single reference noisy image to any clean image. The method learns a yc=γc⋅xc−μcσc2+ϵ+βc,y_c=\gamma_c \cdot \frac{x_c-\mu_c}{\sqrt{\sigma_c^2+\epsilon}}+\beta_c,2-dimensional noise embedding with a contrastive encoder branch yc=γc⋅xc−μcσc2+ϵ+βc,y_c=\gamma_c \cdot \frac{x_c-\mu_c}{\sqrt{\sigma_c^2+\epsilon}}+\beta_c,3, using positives from the same noise distribution and negatives from different distributions. A momentum key encoder yc=γc⋅xc−μcσc2+ϵ+βc,y_c=\gamma_c \cdot \frac{x_c-\mu_c}{\sqrt{\sigma_c^2+\epsilon}}+\beta_c,4 and a MoCo-style queue of size yc=γc⋅xc−μcσc2+ϵ+βc,y_c=\gamma_c \cdot \frac{x_c-\mu_c}{\sqrt{\sigma_c^2+\epsilon}}+\beta_c,5 support InfoNCE training with temperature yc=γc⋅xc−μcσc2+ϵ+βc,y_c=\gamma_c \cdot \frac{x_c-\mu_c}{\sqrt{\sigma_c^2+\epsilon}}+\beta_c,6:

yc=γc⋅xc−μcσc2+ϵ+βc,y_c=\gamma_c \cdot \frac{x_c-\mu_c}{\sqrt{\sigma_c^2+\epsilon}}+\beta_c,7

The full architecture contains a discriminator with two branches, yc=γc⋅xc−μcσc2+ϵ+βc,y_c=\gamma_c \cdot \frac{x_c-\mu_c}{\sqrt{\sigma_c^2+\epsilon}}+\beta_c,8 and yc=γc⋅xc−μcσc2+ϵ+βc,y_c=\gamma_c \cdot \frac{x_c-\mu_c}{\sqrt{\sigma_c^2+\epsilon}}+\beta_c,9, and a generator Ilow∈RH×W×3I_{\text{low}} \in \mathbb{R}^{H \times W \times 3}00 conditioned on a clean image Ilow∈RH×W×3I_{\text{low}} \in \mathbb{R}^{H \times W \times 3}01 and the reference-noise embedding Ilow∈RH×W×3I_{\text{low}} \in \mathbb{R}^{H \times W \times 3}02. The generator outputs a synthesized noisy image Ilow∈RH×W×3I_{\text{low}} \in \mathbb{R}^{H \times W \times 3}03. Training combines contrastive, adversarial, feature-matching, and Gaussian-filtered reconstruction losses. The discriminator objective is

Ilow∈RH×W×3I_{\text{low}} \in \mathbb{R}^{H \times W \times 3}04

and the generator objective is

Ilow∈RH×W×3I_{\text{low}} \in \mathbb{R}^{H \times W \times 3}05

with Ilow∈RH×W×3I_{\text{low}} \in \mathbb{R}^{H \times W \times 3}06 (Lee et al., 2023).

Training uses SIDD-Medium, SIDD validation, SIDD+, DND, DIV2K, and BSDS500. Synthetic noise includes Gaussian with Ilow∈RH×W×3I_{\text{low}} \in \mathbb{R}^{H \times W \times 3}07, Poisson with Ilow∈RH×W×3I_{\text{low}} \in \mathbb{R}^{H \times W \times 3}08, and Poisson-Gaussian mixtures over those ranges. Mini-batches contain Ilow∈RH×W×3I_{\text{low}} \in \mathbb{R}^{H \times W \times 3}09 patches of size Ilow∈RH×W×3I_{\text{low}} \in \mathbb{R}^{H \times W \times 3}10, with Ilow∈RH×W×3I_{\text{low}} \in \mathbb{R}^{H \times W \times 3}11 real SIDD patches and Ilow∈RH×W×3I_{\text{low}} \in \mathbb{R}^{H \times W \times 3}12 synthetic patches. Optimization uses Adam with learning rate Ilow∈RH×W×3I_{\text{low}} \in \mathbb{R}^{H \times W \times 3}13, Ilow∈RH×W×3I_{\text{low}} \in \mathbb{R}^{H \times W \times 3}14, Ilow∈RH×W×3I_{\text{low}} \in \mathbb{R}^{H \times W \times 3}15, and Ilow∈RH×W×3I_{\text{low}} \in \mathbb{R}^{H \times W \times 3}16 regularization Ilow∈RH×W×3I_{\text{low}} \in \mathbb{R}^{H \times W \times 3}17. The model is trained for Ilow∈RH×W×3I_{\text{low}} \in \mathbb{R}^{H \times W \times 3}18 epochs, with discriminator and generator each updated Ilow∈RH×W×3I_{\text{low}} \in \mathbb{R}^{H \times W \times 3}19 times per epoch, taking approximately one week on two Tesla V100 GPUs (Lee et al., 2023).

The reported results concern both noise-generation fidelity and downstream denoising. On SIDD validation, NoiseTransfer achieves AKLD/KS of Ilow∈RH×W×3I_{\text{low}} \in \mathbb{R}^{H \times W \times 3}20, compared with Ilow∈RH×W×3I_{\text{low}} \in \mathbb{R}^{H \times W \times 3}21 for DANet and Ilow∈RH×W×3I_{\text{low}} \in \mathbb{R}^{H \times W \times 3}22 for CycleISP; on SIDD+, it reports Ilow∈RH×W×3I_{\text{low}} \in \mathbb{R}^{H \times W \times 3}23. When RIDNet is trained on NoiseTransfer-generated pairs, the reported denoising performance is Ilow∈RH×W×3I_{\text{low}} \in \mathbb{R}^{H \times W \times 3}24 on SIDD validation, Ilow∈RH×W×3I_{\text{low}} \in \mathbb{R}^{H \times W \times 3}25 on SIDD+, and Ilow∈RH×W×3I_{\text{low}} \in \mathbb{R}^{H \times W \times 3}26 on DND, approaching the corresponding ground-truth-trained references and exceeding CycleISP in the reported comparisons (Lee et al., 2023).

The broader connection to Noise2Contrast lies in what is being contrasted: noisy images from the same versus different noise distributions. This differs from the tomographic setting, where the contrast is a paired acquisition of the same anatomy under different imaging settings, and from NoiSER, where the operative object is local adjacent-pixel contrast reconstructed through self-regression. The paper also reports failure of ablations that remove the discriminator-side noise contrastive loss: removing Ilow∈RH×W×3I_{\text{low}} \in \mathbb{R}^{H \times W \times 3}27 causes divergence by epoch Ilow∈RH×W×3I_{\text{low}} \in \mathbb{R}^{H \times W \times 3}28, while adding Ilow∈RH×W×3I_{\text{low}} \in \mathbb{R}^{H \times W \times 3}29 and Ilow∈RH×W×3I_{\text{low}} \in \mathbb{R}^{H \times W \times 3}30 stabilizes training and yields the best AKLD/KS (Lee et al., 2023).

5. Fine-grained RAW noise estimation via contrastive learning

The paper "Estimating Fine-Grained Noise Model via Contrastive Learning" is not officially named Noise2Contrast, and the data explicitly notes that the nickname is not used by the authors (Zou et al., 2022). Nevertheless, it is closely related conceptually because it uses contrastive learning to isolate noise statistics rather than scene content and then synthesizes realistic noise for downstream denoiser training.

The method operates in RAW space and models noise with a fine-grained, physics-based parameterization

Ilow∈RH×W×3I_{\text{low}} \in \mathbb{R}^{H \times W \times 3}31

combining Poisson shot noise, Gaussian readout noise with non-zero mean, Gaussian row noise, and a multiplicative gain. With clean RAW Ilow∈RH×W×3I_{\text{low}} \in \mathbb{R}^{H \times W \times 3}32, the generative form is

Ilow∈RH×W×3I_{\text{low}} \in \mathbb{R}^{H \times W \times 3}33

where Ilow∈RH×W×3I_{\text{low}} \in \mathbb{R}^{H \times W \times 3}34 and Ilow∈RH×W×3I_{\text{low}} \in \mathbb{R}^{H \times W \times 3}35. Under a heteroscedastic Gaussian approximation, the noise level function becomes

Ilow∈RH×W×3I_{\text{low}} \in \mathbb{R}^{H \times W \times 3}36

To emphasize noise statistics, the input noisy RAW image is first transformed by a Haar wavelet transform Ilow∈RH×W×3I_{\text{low}} \in \mathbb{R}^{H \times W \times 3}37. A ResNet encoder and MLP projection head are trained with InfoNCE-style contrastive learning, where positives share the same noise parameters but differ in scene content, and negatives use different parameter tuples. A regression head predicts Ilow∈RH×W×3I_{\text{low}} \in \mathbb{R}^{H \times W \times 3}38, and the full objective combines weighted MSE on transformed parameters with a contrastive term of weight Ilow∈RH×W×3I_{\text{low}} \in \mathbb{R}^{H \times W \times 3}39. Training uses Adam with Ilow∈RH×W×3I_{\text{low}} \in \mathbb{R}^{H \times W \times 3}40, learning rate Ilow∈RH×W×3I_{\text{low}} \in \mathbb{R}^{H \times W \times 3}41 decayed by Ilow∈RH×W×3I_{\text{low}} \in \mathbb{R}^{H \times W \times 3}42 every Ilow∈RH×W×3I_{\text{low}} \in \mathbb{R}^{H \times W \times 3}43 epochs, batch size Ilow∈RH×W×3I_{\text{low}} \in \mathbb{R}^{H \times W \times 3}44, and Ilow∈RH×W×3I_{\text{low}} \in \mathbb{R}^{H \times W \times 3}45 epochs on an NVIDIA RTX 3090 (Zou et al., 2022).

The estimator is then applied to noisy test images from a target camera, without calibration frames or paired data, to fit camera-level relationships such as

Ilow∈RH×W×3I_{\text{low}} \in \mathbb{R}^{H \times W \times 3}46

and optionally Ilow∈RH×W×3I_{\text{low}} \in \mathbb{R}^{H \times W \times 3}47 with ISO Ilow∈RH×W×3I_{\text{low}} \in \mathbb{R}^{H \times W \times 3}48. These inferred relationships define a camera-specific sampling distribution for realistic RAW noise synthesis. The intended use is to generate synthetic noisy/clean pairs for denoiser training, or to provide a non-blind denoiser with an estimated noise level function (Zou et al., 2022).

The reported synthesis fidelity on SIDD is strong: average KL divergence between noise histograms is Ilow∈RH×W×3I_{\text{low}} \in \mathbb{R}^{H \times W \times 3}49 for the proposed method, compared with Ilow∈RH×W×3I_{\text{low}} \in \mathbb{R}^{H \times W \times 3}50 for AWGN, Ilow∈RH×W×3I_{\text{low}} \in \mathbb{R}^{H \times W \times 3}51 for Poisson-Gaussian, Ilow∈RH×W×3I_{\text{low}} \in \mathbb{R}^{H \times W \times 3}52 for NoiseFlow, and Ilow∈RH×W×3I_{\text{low}} \in \mathbb{R}^{H \times W \times 3}53 for CANGAN. On SIDD S6 denoising with a U-Net trained on synthesized pairs, the reported “All ISOs” result is Ilow∈RH×W×3I_{\text{low}} \in \mathbb{R}^{H \times W \times 3}54, compared with Ilow∈RH×W×3I_{\text{low}} \in \mathbb{R}^{H \times W \times 3}55 for paired real data; the ablation without contrastive loss drops to Ilow∈RH×W×3I_{\text{low}} \in \mathbb{R}^{H \times W \times 3}56 (Zou et al., 2022).

6. Common principles, distinctions, and recurring limitations

Taken together, these works suggest that Noise2Contrast is best understood as a family resemblance across different imaging problems rather than a single canonical recipe. The unifying principle is that noise contains exploitable structure. In NoiSER, the structure is the abundance of local differences in Gaussian noise coupled with instance normalization and a gray-world bias (Zhang et al., 2022). In tomographic denoising, the structure is the independence of noise across co-registered imaging contrasts, which turns another noisy contrast into a valid self-supervised target (Wagner et al., 2022). In NoiseTransfer and the RAW noise model paper, the structure is distributional: noisy images or synthesized parameter-matched samples can be contrasted so that the representation becomes sensitive to the underlying noise process rather than scene semantics (Lee et al., 2023, Zou et al., 2022).

The distinctions are equally important. NoiSER is a compact model trained once on synthetic Gaussian noise and then applied broadly, not a per-image fitting procedure. The tomographic Noise2Contrast method is not a generic contrastive-learning framework but a denoiser-plus-transfer architecture whose validity depends on cross-contrast independence and anatomical alignment. NoiseTransfer is a generative model for producing realistic noisy images from a single noisy reference. The fine-grained RAW method is a parameter estimator and synthesis pipeline rather than a restoration network (Zhang et al., 2022, Wagner et al., 2022, Lee et al., 2023, Zou et al., 2022).

The recurring failure modes are also modality-specific. Gray-world assumptions can over-neutralize genuine color casts in low-light enhancement; misregistration and imperfect domain transfer can bias tomographic denoising; rare or highly structured artifacts may be underrepresented in learned noise embeddings for noise generation; and RAW-domain physics-based synthesis does not model the ISP or non-Bayer CFA variants (Zhang et al., 2022, Wagner et al., 2022, Lee et al., 2023, Zou et al., 2022). A plausible implication is that future uses of the term will continue to diverge unless authors specify precisely what is being contrasted: image contrasts, adjacent-pixel contrasts, noisy-distribution embeddings, or noise-parameter classes.

Topic to Video (Beta)

No one has generated a video about this topic yet.

Whiteboard

No one has generated a whiteboard explanation for this topic yet.

Follow Topic

Get notified by email when new papers are published related to Noise2Contrast.