---
title: 'Median2Median: Zero-Shot Structured Noise Denoising'
url: https://www.emergentmind.com/topics/median2median-m2m
type: topic
---

# Median2Median: Zero-Shot Structured Noise Denoising

Searching arXiv for the named Median2Median paper and closely related median-based methods to ground the article in the provided literature.
arxiv_search:
{"query":"ti:\"Median2Median\" OR abs:\"Median2Median\"","max_results":5,"sort_by":"submittedDate","sort_order":"descending"}
arxiv_search:
{"query":"id:2510.01666 OR id:1410.6604 OR id:2211.08131","max_results":10,"sort_by":"relevance","sort_order":"descending"}
Median2Median (M2M) is a blind zero-shot denoising framework for the case in which a single noisy image is corrupted by structured, directionally correlated noise rather than only by i.i.d. perturbations. It requires no clean target, repeated acquisition, or external training set. Its central mechanism is the construction of pseudo-independent sub-image pairs from one noisy image by combining directional interpolation, generalized median filtering, and randomized assignment, after which the resulting pairs are used for Noise2Noise-style training. The method is presented as a zero-shot denoising framework beyond the strict i.i.d. assumption, with realistic simulation studies reporting performance on par with state-of-the-art zero-shot methods under i.i.d. noise and consistently stronger performance under correlated noise [2510.01666].

## 1. Problem regime and conceptual aim

M2M addresses denoising under structured noise with strong anisotropy and spatial correlation. In the formulation used by the paper, the target setting is not generic additive pixelwise noise but corruption with a dominant direction and nontrivial covariance across neighboring pixels. The motivating examples include banding, striping, and weak oriented artifacts in microscopy, CT, and related imaging systems. This matters because earlier zero-shot methods, including Noise2Fast (N2F) and Zero-Shot Noise2Noise (ZS-N2N), construct training pairs from local neighborhoods in ways that work when noise is approximately independent across pixels, but they degrade when neighboring samples inherit the same directional artifact [2510.01666].

The paper positions M2M between two established regimes. Supervised or otherwise data-driven denoisers can often remove structured noise when trained on representative data, but they depend on clean/noisy or noisy/noisy datasets and may generalize poorly outside the training distribution. Existing zero-shot methods avoid external data and are robust to distribution shift because training occurs on the test image itself, but their pair construction implicitly assumes i.i.d. zero-mean noise. M2M is proposed to close that gap by manufacturing from a single image a set of paired observations that are not strictly independent, but are sufficiently de-structured for Noise2Noise-style learning to be effective in practice [2510.01666].

## 2. Sampling architecture and median-centered pair construction

The M2M pipeline begins by reflect-padding the noisy image so that its dimensions are multiples of \(3\), then partitioning the padded image into non-overlapping \(3\times 3\) patches. For each of the nine relative positions in a patch—TL, T, TR, L, C, R, BL, B, BR—the method processes that same position across all patches, thereby forming a sub-image. The paper emphasizes this block-wise sampling rather than stride-1 sliding windows because non-overlapping patches reduce overlap-induced dependencies [2510.01666].

For a target pixel at \((i,j)\), M2M builds a local candidate set from directional estimates in a \(3\times 3\) interpolation window. In zero-order directional interpolation, neighboring pixels are used directly,
\[
\hat{I}_{(\Delta i, \Delta j)}(i, j) = \tilde{I}(i + \Delta i, j + \Delta j),
\]
with either
\[
\mathcal{N}_4 = \{(\pm 1, 0), (0, \pm 1)\}
\]
or
\[
\mathcal{N}_8 = \mathcal{N}_4 \cup \{(\pm 1, \pm 1)\}.
\]
This yields the variants M2M-0 (4N) and M2M-0 (8N). In first-order directional interpolation, the method averages symmetric neighbors along four orientations,
\[
\hat{I}_\theta(i, j) = \frac{1}{2} \left[ \tilde{I}(i + \Delta i_\theta, j + \Delta j_\theta) + \tilde{I}(i - \Delta i_\theta, j - \Delta j_\theta) \right],
\]
for \(\theta \in \{0^\circ,45^\circ,90^\circ,135^\circ\}\), giving the M2M-1 variant [2510.01666].

The current pixel is always included in the candidate set. For zero-order interpolation,
\[
\mathcal{S}_0(i, j) = \left\{ \tilde{I}(i, j) \right\} \cup \left\{ \hat{I}_{(\Delta i, \Delta j)}(i, j) \mid (\Delta i, \Delta j) \in \mathcal{N}_k \right\}, \quad k \in \{4, 8\},
\]
and for first-order interpolation,
\[
\mathcal{S}_1(i, j) = \left\{ \tilde{I}(i, j) \right\} \cup \left\{ \hat{I}_\theta(i, j) \mid \theta \in \{0^\circ,45^\circ,90^\circ,135^\circ\} \right\}.
\]
After interpolation, the candidates are sorted,
\[
v_{(1)} \leq v_{(2)} \leq \cdots \leq v_{(n)},
\]
and M2M retains the three values nearest the median,
\[
\mathcal{S}_f(i,j) = \{v_{(\lceil n/2 \rceil -1)},\ v_{(\lceil n/2 \rceil)},\ v_{(\lceil n/2 \rceil +1)}\}.
\]
The paper calls this a generalized median filtration. Its purpose is not conventional smoothing, but adaptive rejection of values distorted by structured artifacts. When anisotropic noise biases one orientation more strongly than others, the values farthest from the median are treated as the most likely directional outliers [2510.01666].

## 3. Randomized assignment and statistical rationale

Once the filtered set \(\mathcal{S}_f(i,j)\) is available, M2M forms paired observations by a randomized assignment strategy (RAS). Two distinct values are sampled uniformly,
\[
(m_1, m_2) \sim \mathcal{U}_2\!\left(\mathcal{S}_f(i,j)\right),
\]
and then their order is randomized,
\[
(x_1,\ x_2)_{r,c} =
\begin{cases}
(m_1,\ m_2), & \text{if } p < 0.5, \\
(m_2,\ m_1), & \text{otherwise},
\end{cases}
\qquad p \sim \mathcal{U}[0,1).
\]
The paper gives two reasons for this step: bias removal and sampling diversity. Fixed ordering would make one sampled image systematically darker or brighter than the other, violating the symmetric assumptions of Noise2Noise-style learning. The randomized swap restores equality of expected intensity between the two sampled images. The same mechanism enlarges the effective sample space; for each pixel pair, the paper states that there are effectively three distinct sampling outcomes,
\[
\binom{3}{2} \times \binom{2}{1} \times \frac{1}{2},
\]
and if the padded image is partitioned into \(N\) patches, each sampling position has a space of size
\[
3^N.
\]
This is presented as a mechanism for avoiding systematic bias while increasing pair diversity [2510.01666].

The statistical argument for M2M is constructive rather than theorem-driven. The synthetic structured-noise model begins from
\[
x \sim \mathcal{N}(0, \sigma_0^2 I),
\]
and generates directional correlated noise by
\[
y = x \ast h_\theta,
\qquad
h_\theta = \Big[\tfrac{1}{\ell}, \tfrac{1}{\ell}, \dots, \tfrac{1}{\ell}\Big] \in \mathbb{R}^{1 \times \ell}.
\]
More explicitly,
\[
y(i,j) = \frac{1}{\ell}\sum_{k=-r}^{r} x\!\big(i+k\,p,\ j+k\,q\big), \quad r=\frac{\ell-1}{2}.
\]
The variance is
\[
\mathrm{Var}(y(i,j)) = \frac{1}{\ell}\sigma_0^2,
\]
and the covariance along the correlated direction is
\[
\mathrm{Cov}\big(y(i,j),\, y(i+t\,p,\, j+t\,q)\big) =
\begin{cases}
\frac{\ell - |t|}{\ell^2}\,\sigma_0^2, & |t| < \ell, \\[6pt]
0, & |t| \geq \ell ,
\end{cases}
\]
while the orthogonal covariance remains
\[
\mathrm{Cov}\big(y(i,j),\, y(i+t(-q),\, j+t\,p)\big) = 0.
\]
Hence the correlation coefficient is
\[
\rho_\theta(t) =
\frac{\mathrm{Cov}(y(i,j),\, y(i+t\,p,\, j+t\,q))}
{\mathrm{Var}(y(i,j))}
=
\begin{cases}
\frac{\ell - |t|}{\ell}, & |t| < \ell, \\[6pt]
0, & |t| \geq \ell .
\end{cases}
\]
The intended implication is that naive neighborhood-based pairing retains the same directional artifact in both elements of a pair, whereas directional interpolation followed by median-centered selection can weaken that shared component. The paper therefore characterizes the resulting observations as pseudo-independent or de-structured noisy pairs rather than formally independent samples [2510.01666].

## 4. Network design, loss functions, and inference procedure

M2M uses nine independent lightweight CNNs, one for each of the nine sampling positions. Each network has three layers: a \(3\times 3\) convolution with stride \(1\) and \(72\) channels followed by PReLU, a second \(3\times 3\) convolution with stride \(1\) and \(72\) channels followed by PReLU, and a \(1\times 1\) convolutional output layer. The architecture is intentionally small because, as the paper notes, deeper models may overfit in zero-shot settings [2510.01666].

The training objective is an extension of the standard Noise2Noise mean squared error. The paper gives a symmetric loss,
\[
\mathcal{L}_{\text{sym}}
=
\frac{1}{2}
\mathbb{E}_{(x_1, x_2)\sim I}
\left[
\| f_\theta(x_1) - x_2 \|_2^2
+
\| f_\theta(x_2) - x_1 \|_2^2
\right],
\]
which trains both directions of each sampled pair. It also introduces a consistency loss. A de-structured image \(\hat I\) is reconstructed by averaging each sampled pair into one sub-image per position and then reassembling the nine sub-images. The denoiser is required to behave consistently whether sampling occurs before or after denoising:
\[
\mathcal{L}_{\text{cons}}(\theta)
=
\tfrac{1}{2}\,\mathbb{E}_{(x_1, x_2)\sim I}
\left[
\| \hat{y}_1 - f_\theta(x_1) \|_2^2
+
\| \hat{y}_2 - f_\theta(x_2) \|_2^2
\right].
\]
The total loss is
\[
\mathcal{L}_{\text{total}}(\theta)
=
\mathcal{L}_{\text{sym}}(\theta)
+
\lambda\,\mathcal{L}_{\text{cons}}(\theta).
\]
Operationally, zero-shot means that no external dataset and no pretraining are used; the networks are trained directly on the single noisy test image. In each epoch, new sub-image pairs are resampled, and the nine networks are trained in parallel. At inference, the sub-images are processed by their corresponding networks, the sampling-denoising procedure is repeated \(k\) times because of the randomized construction, repeated outputs are averaged for convergence, and the final denoised image is obtained by reassembling the nine outputs [2510.01666].

## 5. Variants, experiments, and reported performance

The experiments use grayscale images from the Kodak dataset and a fluorescence microscopy dataset, with synthetic noise that is i.i.d. Gaussian when \(\ell=1\) and structured directional correlated noise when \(\ell>1\). Evaluation is reported in terms of PSNR and SSIM. The baselines are Noise2Fast (N2F), Zero-Shot Noise2Noise (ZS-N2N), and BM3D, with BM3D treated as non-blind and given the true noise variance [2510.01666].

Under i.i.d. noise, the paper reports that M2M is broadly competitive with state-of-the-art blind zero-shot methods. At low noise it may be slightly below BM3D and N2F, whereas at higher noise levels it often becomes the strongest blind zero-shot method and can in some cases exceed BM3D. One reported fluorescence microscopy result for \(\ell=1,\ \sigma_n=0.15\) is:
- BM3D: \(29.29 / 0.7513\)
- N2F: \(27.54 / 0.7106\)
- ZS-N2N: \(27.28 / 0.6799\)
- M2M-0 (4N): \(29.68 / 0.7231\)
- M2M-0 (8N): \(30.54 / 0.7154\)
- M2M-1: \(29.27 / 0.7364\)

The more distinctive behavior appears under correlated noise. For \(\ell=3,\ \sigma_n=0.10\), the reported fluorescence microscopy scores are:
- BM3D: \(28.21 / 0.6082\)
- N2F: \(27.10 / 0.4983\)
- ZS-N2N: \(24.18 / 0.3194\)
- M2M-0 (4N): \(29.61 / 0.6587\)
- M2M-0 (8N): \(29.43 / 0.6495\)
- M2M-1: \(30.25 / 0.7314\)

For \(\ell=5,\ \sigma_n=0.15\), the same trend persists:
- BM3D: \(23.00 / 0.3512\)
- N2F: \(22.77 / 0.3022\)
- ZS-N2N: \(19.75 / 0.1616\)
- M2M-0 (4N): \(24.97 / 0.4116\)
- M2M-0 (8N): \(25.23 / 0.4461\)
- M2M-1: \(26.12 / 0.5594\)

The paper interprets these results as evidence that prior zero-shot pairings still retain the directional artifact, while M2M’s median-centered pair construction is better aligned with correlated-noise suppression. Variant behavior is also differentiated: zero-order interpolation tends to preserve more fine detail, whereas first-order interpolation tends to suppress structured noise more strongly. This is consistent with the benchmark table, where M2M-1 is often strongest on correlated noise. The ablation study removes block-wise sampling, inclusion of the center pixel in the candidate set, randomized assignment, and repeated inference one at a time, and the paper concludes visually that every component is important [2510.01666].

## 6. Assumptions, limitations, and scope of applicability

M2M is built around several explicit assumptions. Structured noise should be sufficiently directional or anisotropic so that some interpolation directions are less corrupted than others. The median-centered candidate set must still contain usable signal estimates; if all directions are similarly corrupted, the method may help less. The pseudo-independence argument is heuristic rather than formally proved. The construction is local and specifically organized around \(3\times 3\) directional structure, so more complex nonlocal or highly irregular structured artifacts are not fully characterized within the paper’s analysis [2510.01666].

The implementation also introduces computational and empirical scope constraints. M2M trains nine networks and uses repeated inference, and the reported denoising time for same-sized noisy images is about \(6\times\) that of N2F or ZS-N2N. The experiments are limited to simulated grayscale noise rather than extensive real-noise validation, so robustness to all real structured noise types is not established. The paper nevertheless suggests applicability to broader imaging problems, explicitly including medical imaging and fluorescence microscopy, and it frames the method as a first step toward effective zero-shot denoising beyond the strict i.i.d. assumption [2510.01666].

## 7. Terminological ambiguity and relation to adjacent work

The name Median2Median should be distinguished from several unrelated uses of the acronym “M2M.” In other parts of the literature, M2m denotes “Major-to-minor Translation” for imbalanced classification rather than any median-based denoising procedure [2004.00431]. In communications and networking, M2M commonly denotes machine-to-machine systems rather than a denoising framework [2209.14427], [1401.4721]. In galactic dynamics, M2M refers to made-to-measure modelling [1210.5521]. These usages are terminologically adjacent but conceptually separate from Median2Median.

Median2Median is also distinct from median-based methods that are related in spirit but not in name or architecture. The MESSAGE estimator in distributed inference applies median aggregation to subset feature-selection indicators and then averages coefficients, so it is median selection plus mean coefficient aggregation rather than a pure median-to-median estimator [1410.6604]. A robust model-based clustering method replaces EM mean and covariance updates with the geometric median and the Median Covariation Matrix, which is median-centered robustification at multiple levels but not a method named Median2Median [2211.08131]. This suggests that Median2Median belongs to a broader family of methods that use median-based operations to suppress bias or contamination, while remaining specific in its formulation as a single-image, zero-shot denoising framework for structured noise.

Source: https://www.emergentmind.com/topics/median2median-m2m