---
title: Repeated Upscaling-Downscaling Process
url: https://www.emergentmind.com/topics/repeated-upscaling-downscaling-process-rudp
type: topic
---

# Repeated Upscaling-Downscaling Process

The Repeated Upscaling–Downscaling Process (RUDP) is a methodology for enhancing the quality of high-dimensional signals—primarily images—via iterative cycles of upsampling and downsampling. Its formal structure and application have been rigorously developed for deep learning–based super-resolution, particularly in contexts where high-frequency detail reconstruction and bidirectional mappings between resolutions are paramount [2601.09410, 2203.00911, 1809.09326, 2310.04964, 1907.11341]. The process has emerged as a unifying abstraction connecting classic iterative back-projection, multigrid methods, modern attention-based super-resolution, invertible flows, and cycle-consistency constraints for arbitrary-scale rescaling.

## 1. Mathematical Formalism and Algorithmic Variants

At its core, RUDP is defined by the repeated application of upsampling ($U$) and downsampling ($D$) operators on a low-resolution (LR) feature or image. For a given input $F_{\rm LR}\in\mathbb{R}^{h\times w\times C}$, the canonical $K$-step RUDP cycle is:

\[
F^{(0)} = F_{\rm LR}
\]
For $k=1,2,...,K$:
\[
\begin{aligned}
F^U_k &= U(F^{(k-1)}) \\
H^{(k)}_{\rm D} &= \text{DetailBlock}(F^U_k) \\
H^{(k)}_{\rm SR} &= F^U_k + H^{(k)}_{\rm D} \\
F_{\rm D}^{(k)} &= D(H^{(k)}_{\rm SR}) \\
F^{(k)} &= \mathrm{Concat}(F_{\rm LR}, F_{\rm D}^{(1)},...,F_{\rm D}^{(k)})
\end{aligned}
\]
The output $H^{(K)}_{\rm SR}$ is mapped to the final RGB super-resolved image, and optionally the detail images $\hat{D}^{(k)}$ and intermediate SR outputs $\hat{I}_{\rm SR}^{(k)}$ are also made available [2601.09410].

In bidirectional or arbitrary-scale variants, such as in BAIRNet [2203.00911], the process involves learned invertible mappings $f_{\rm up}$ and $f_{\rm down}$ parameterized for arbitrary scale factors and robust to repeated cycles, with explicit cycle-idempotence objectives.

The process can be instantiated recursively (multi-level, as in multigrid methods [1809.09326]) or in stage-wise, recurrent training loops [1907.11341].

## 2. Integration into Deep Super-Resolution Architectures

RUDP acts as a meta-structuring device for deep super-resolution networks. A typical architecture employing RUDP includes:

* **Feature-Extraction Block (FE):** Processes the LR image to produce a feature tensor, typically via convolution and residual blocks.
* **Upscale Block (UB):** Transposed convolution and additional layers upscale the feature map; a detail block isolates high-frequency structure.
* **Downscale Block (DB):** Strided convolutions (or equivalent operators) reduce the resolution of the upscaled output, generating new inputs for subsequent RUDP iterations.
* **Iterative Pipeline:** The pipeline cycles through $K$ repetitions of UB$\to$DB, each supervised independently (deep supervision) via reconstruction and detail losses [2601.09410, 1809.09326].

Specialized RUDP architectures, e.g., BAIRNet [2203.00911], employ shared encoders and subpixel MLPs to enable scale-continuity and preserve detail through repeated rescaling cycles. In SDFlow [2310.04964], a single invertible normalizing-flow network simultaneously models both LR$\to$HR (super-resolution) and HR$\to$LR (downscaling) by decoupling shared content and domain-specific latent variables.

## 3. Training Objectives, Losses, and Deep Supervision

RUDP-enhanced models are supervised not solely by standard image reconstruction losses but also by high-frequency–focused objectives. A prototypical design is:

* **Reconstruction Loss:** $L_{SR}^{(k)} = \lVert I_{\rm HR} - \hat{I}_{\rm SR}^{(k)} \rVert_1$.
* **Detail Loss:** Using Laplacian-pyramid decomposition, $L_{\rm detail}^{(k)} = \lVert D_{\rm GT} - \hat{D}^{(k)} \rVert_1$.
* **Total Loss:** Weighted sum across RUDP stages,
\[
L_{\rm total} = \sum_{k=1}^K W_k [ L_{SR}^{(k)} + \lambda L_{\rm detail}^{(k)} ]
\]
where $\lambda$ controls the detail-vs-reconstruction tradeoff and $W_k$ sets the relative importance of each RUDP stage [2601.09410].

For arbitrary-scale and cycle-idempotent objectives [2203.00911], loss terms include cycle-consistency, weak LR reference, and robustness to repeated up/down cycles:
\[
\mathcal{L}_{\rm total} = \lambda_1\,\mathcal{L}_{\rm cycle}^{(n)} + \lambda_2\,\mathcal{L}_{\rm ref}
\]
Invertible models (SDFlow [2310.04964]) minimize exact maximum likelihood, latent-space alignment losses, and adversarial terms on the content/structure codes, enabling unpaired, bidirectional learning.

## 4. Theoretical Rationale and Empirical Effects

The motivation for RUDP is both empirical and theoretical:

* **High-Frequency Amplification:** Repeated rescaling, especially when coupled with explicit detail supervision, forces the network to progressively correct high-frequency and error residuals, mitigating over-smoothing typical of pixel-based $L_1$ or MSE-only approaches [2601.09410].
* **Feature Diversity:** Cycling outputs through the network after downscaling exposes subsequent RUDP iterations to previously reconstructed as well as new detail, increasing diversity and promoting robustness against artifacts [2601.09410].
* **Progressive Refinement:** Deep supervision on intermediates stabilizes training, enables incremental error correction, and encourages convergence to sharper outputs [1809.09326].
* **Cycle-Idempotency:** In arbitrary-scale or invertible architectures, explicit multi-cycle losses ensure the RUDP system avoids error accumulation or drift even under repeated application, preserving both numerical fidelity and perceptual realism [2203.00911, 2310.04964].

## 5. Empirical Results and Ablation Studies

Controlled ablations confirm the effectiveness and complementarity of RUDP and detail supervision. For the LaUD network [2601.09410], comparative results on Set5/Set14/BSD100 show:

| Model | RUDP | DetailLoss | Set5 PSNR | Set14 PSNR | BSD100 PSNR |
|-------|------|------------|-----------|------------|-------------|
| M1    | no   | no         | 38.09     | 33.97      | 32.29       |
| M2    | no   | yes        | 38.28     | 34.28      | 32.40       |
| M3    | yes  | no         | 38.32     | 34.61      | 32.49       |
| M4    | yes  | yes        | 38.42     | 34.77      | 32.55       |

RUDP alone improves PSNR (e.g., +0.23 dB on Set5), with further gains when combined with detail loss (+0.34 dB) [2601.09410].

Robustness to repeated cycles is quantitatively demonstrated in BAIRNet [2203.00911], where the model sustains less than 0.5 dB degradation over five RUDP cycles, contrasting with 2–3 dB drops in earlier invertible or unidirectional models, and no visible ghosting or color warps after multiple passes.

Normalizing-flow models (SDFlow [2310.04964]) enable stochastic, many-to-many RUDP cycles, producing diverse yet plausible HR–LR pairs over successive cycles, with stable perceptual metrics and reference-free fidelity scores.

## 6. Relationships to Classical Methods and Unifying Abstractions

RUDP generalizes classic iterative back-projection (IBP) [1809.09326], where an upscaler is corrected by feeding back errors via a downscaling operator, as well as multigrid solvers that recursively transfer corrections across scales. Modern RUDP implementations embed these principles in deep architectures, replacing fixed filters with trainable convolutional or flow-based modules.

In multigrid back-projection super-resolution [1809.09326], RUDP is instantiated as a recursive, learned V- or W-cycle, synthesizing outputs at multiple upscaling factors with minimal parameter counts and per-cycle residual corrections.

The stage-wise RUDP in recurrent SR network training [1907.11341] extends the concept to self-distillation: each SR network is trained against increasingly enhanced HR targets generated via its own previous stages, leading to sharper, more perceptually aligned outcomes.

## 7. Design Considerations, Limitations, and Practical Guidelines

Empirical findings support optimal RUDP repeat counts ($K$) of 3–4, where both PSNR and perceptual metrics (e.g., VIQET MOS [1907.11341]) typically saturate; further cycles yield negligible improvement or slight over-enhancement. Key parameters and recommendations include:

- **Upsampler:** Transposed convolution + cascaded convs with LeakyReLU or tailored MLP subpixel modules [2601.09410, 2203.00911].
- **Downsampler:** Strided convolution with or without learned subpixel weights [2601.09410, 2203.00911].
- **Detail-loss norm:** $L_1$ preferred over $L_2$ for edge/texture fidelity [2601.09410].
- **Cycle weightings:** Later cycles may be given higher loss weights to encourage incremental refinement [2601.09410].

RUDP increases training cost linearly with the number of cycles but imposes negligible additional inference cost if only the final output is required. In bidirectional, invertible, or stochastic settings, RUDP supports arbitrary or indefinite cycling (with stochastic sampling in the latent space yielding diverse outputs [2310.04964]).

Convergence is practical but not theoretically guaranteed for arbitrary nonlinear architectures; empirical saturation is typically reached by $K=3$–4 [1907.11341].

---

**References:**

- "Detail Loss in Super-Resolution Models Based on the Laplacian Pyramid and Repeated Upscaling and Downscaling Process" [2601.09410]
- "Towards Bidirectional Arbitrary Image Rescaling: Joint Optimization and Cycle Idempotence" [2203.00911]
- "Multigrid Backprojection Super-Resolution and Deep Filter Visualization" [1809.09326]
- "Learning Many-to-Many Mapping for Unpaired Real-World Image Super-resolution and Downscaling" [2310.04964]
- "Image Enhancement by Recurrently-trained Super-resolution Network" [1907.11341]

Source: https://www.emergentmind.com/topics/repeated-upscaling-downscaling-process-rudp