---
title: Residual-based Efficient Bidirectional Diffusion Model
url: https://www.emergentmind.com/topics/residual-based-efficient-bidirectional-diffusion-model-rbdm
type: topic
---

# Residual-based Efficient Bidirectional Diffusion Model

Searching arXiv for the cited RBDM-related papers to ground the article and confirm bibliographic details.
arxiv_search.query({"search_query":"id:2403.07319 OR id:2503.01352 OR id:2508.11134 OR id:2510.23116","max_results":10,"sort_by":"submittedDate","sort_order":"descending"})
I’m going to retrieve the relevant arXiv entries for the exact identifiers and titles to ensure the terminology and chronology are aligned.
arxiv_search.search({"query":"2403.07319 2503.01352 2508.11134 2510.23116 residual-based efficient bidirectional diffusion model"})
Residual-based Efficient Bidirectional Diffusion Model (RBDM) denotes a class of conditional diffusion formulations that replace the conventional image-to-Gaussian trajectory with short, residual-aware transitions between paired image domains. In one established usage, the term is an accurate conceptual description of ResShift, which performs image restoration by shifting the residual between a high-quality (HQ) image and its low-quality (LQ) counterpart along a short Markov chain [2403.07319]. In a later and explicit usage, it names a dual-chain model for image dehazing and haze generation, where residuals between haze-free and hazy images govern bidirectional conditional diffusion [2508.11134]. Across these formulations, the defining idea is that the forward process is anchored to the observed degradation domain rather than to pure Gaussian noise, so the reverse process traverses a shorter and more task-aligned path.

## 1. Terminology, scope, and conceptual definition

Within image restoration, an RBDM is characterized by three coupled properties. It is **residual-based** because the diffusion trajectory is parameterized by the difference between paired domains, such as \( \bm{e}_0 = \bm{y}_0 - \bm{x}_0 \) for LQ–HQ image pairs. It is **efficient** because the forward process terminates near the observed degraded image rather than at white noise, which reduces the number of reverse sampling steps. It is **bidirectional** because there is an explicit forward transition from one endpoint distribution to the other and a learned reverse transition that inverts this path [2403.07319, 2508.11134].

The term is not fully canonical across the literature. In the ResShift paper, “Residual-based Efficient Bidirectional Diffusion Model” is not the exact title used by the authors, but it is an accurate conceptual description of the method. By contrast, “RBDM” is also used as an acronym for **Regulated Bridge Diffusion Model** in polarization-based virtual staining, where the model is a Brownian-bridge construction rather than a residual-shifting image restoration model [2503.01352]. A related but differently named development is the **Residual Diffusion Bridge Model (RDBM)**, which reformulates generalized diffusion bridges with residual-modulated noise injection for universal image restoration [2510.23116].

A common misconception is that all conditional diffusion models for restoration are already residual-based and bidirectional in this stronger sense. The cited work distinguishes RBDM-style constructions from standard conditional diffusion pipelines that preserve the classic DDPM forward process to noise and merely condition the reverse process on degraded observations. In the residual-shifting view, the paired domains themselves define the bridge, and the residual becomes the task-specific direction of transport [2403.07319].

## 2. Residual shifting between paired image domains

The clearest early formulation appears in ResShift. Let \( \bm{x}_0 \) denote the HQ image and \( \bm{y}_0 \) the LQ image at the same resolution, and define the residual
\[
\bm{e}_0 = \bm{y}_0 - \bm{x}_0.
\]
The forward diffusion is a Markov chain \( \bm{x}_0 \to \bm{x}_1 \to \cdots \to \bm{x}_T \) conditioned on \( \bm{y}_0 \), with transition kernel
\[
q(\bm{x}_t \mid \bm{x}_{t-1}, \bm{y}_0)
= \mathcal{N}\bigl(\bm{x}_t;\; \bm{x}_{t-1} + \alpha_t \bm{e}_0,\; \kappa^2 \alpha_t \bm{I}\bigr).
\]
Its marginal form is
\[
q(\bm{x}_t \mid \bm{x}_0, \bm{y}_0)
= \mathcal{N}\bigl(\bm{x}_t;\; \bm{x}_0 + \eta_t \bm{e}_0,\; \kappa^2 \eta_t \bm{I}\bigr),
\]
which yields the reparameterization
\[
\bm{x}_t
= (1-\eta_t)\bm{x}_0 + \eta_t \bm{y}_0 + \kappa \sqrt{\eta_t}\,\bm{\xi}_t.
\]
The mean is therefore a convex combination of HQ and LQ endpoints, while the variance is controlled independently by \( \kappa \) and the schedule \( \{\eta_t\} \) [2403.07319].

This construction changes the geometry of the diffusion path. At \( t=1 \), with \( \eta_1 \to 0 \), the state remains close to the HQ image. At \( t=T \), with \( \eta_T \approx 1 \), the terminal distribution is a perturbation around the LQ image rather than around pure Gaussian noise. The forward process therefore moves from HQ to a noisy LQ prior, and the reverse process starts from that noisy LQ prior and restores the HQ image. The bidirectionality is thus defined in residual space: forward diffusion shifts along \( \bm{y}_0 - \bm{x}_0 \), and reverse denoising learns to invert that shift [2403.07319].

The reverse kernel is parameterized as a Gaussian whose mean depends on a network prediction of the clean image \( \bm{x}_0 \), not on direct noise prediction. With fixed posterior variance, the ELBO simplifies to an \( L_2 \)-style loss on the prediction \( f_{\bm{\theta}}(\bm{x}_t,\bm{y}_0,t) \approx \bm{x}_0 \). In practice, the weighted form is dropped and an unweighted \( L_2 \) loss is used, and a perceptual LPIPS term is added because pure \( L_2 \) leads to average solutions and oversmoothing, especially when \( T < 5 \) [2403.07319].

The schedule is equally central. ResShift enforces \( \kappa \sqrt{\eta_1} \lesssim 0.04 \), sets
\[
\eta_1 = \min\left(\left(\frac{0.04}{\kappa}\right)^2,\; 0.001\right),
\quad
\eta_T = 0.999,
\]
and uses a non-uniform geometric schedule for intermediate steps. The parameter \( p \) controls how quickly \( \sqrt{\eta_t} \) grows: smaller \( p \) yields smoother residual shifting at early steps, while larger \( p \) yields more aggressive residual change. This explicit separation between shifting speed and noise strength is a defining feature of the residual-based efficient formulation [2403.07319].

## 3. Dual Markov chains for dehazing and haze generation

The 2025 dehazing paper uses the name **Residual-based Efficient Bidirectional Diffusion Model** directly and extends the residual-shifting principle to two symmetric conditional distributions: haze-free \( \leftrightarrow \) hazy [2508.11134]. Let \( x_0 \) denote the haze-free image and \( y_0 \) the hazy image. Two residuals are defined:
\[
e_0 = y_0 - x_0,
\qquad
e_1 = x_0 - y_0.
\]
The model then constructs **dual Markov chains**. The dehazing chain starts from \( x_0 \) and drifts toward \( y_0 \),
\[
q(x_{t_x} \mid x_{t_x-1}, y_{t_y})
=
\mathcal{N}\!\left(x_{t_x};\; x_{t_x-1} + \alpha_{t_x} e_0,\; \kappa^2 \alpha_{t_x} I\right),
\]
while the haze-generation chain starts from \( y_0 \) and drifts toward \( x_0 \),
\[
q(y_{t_y} \mid y_{t_y-1}, x_{t_x})
=
\mathcal{N}\!\left(y_{t_y};\; y_{t_y-1} + \alpha_{t_y} e_1,\; \kappa^2 \alpha_{t_y} I\right).
\]
In both directions, the terminal state is a Gaussian perturbation around the opposite endpoint rather than an unconditional noise prior [2508.11134].

The reverse processes are likewise paired. One chain models \( p(x_0 \mid y_{t_y}) \) for dehazing, and the other models \( p(y_0 \mid x_{t_x}) \) for haze generation. The reverse kernels are Gaussian and share a unified network \( f_\theta(x_{t_x}, y_{t_y}, t_x, t_y) \), which parameterizes the means in both directions. The theoretical KL objectives reduce to a unified \( L_1 \)-type regression loss with constant weight \( \omega = 1 \), and the paper reports that this unweighted form performs better than more sophisticated weighting [2508.11134].

A notable design choice is patch-based training and inference. The network is trained on randomly cropped patches defined by binary masks and `Crop`, rather than on full images. At test time, arbitrary-resolution images are decomposed into overlapping patches, each patch is processed independently through the diffusion update, and overlapping predictions are averaged pixelwise:
\[
\text{Output}(p) = \frac{Sample(p)}{M(p)}.
\]
This makes the model size-agnostic, reduces memory and computation, and increases effective data diversity on relatively small dehazing datasets [2508.11134].

The resulting RBDM is explicitly bidirectional in a stronger operational sense than ResShift. The same model supports dehazing and haze generation, uses only 15 sampling steps, and implements smooth transitions between haze-free and hazy domains. The paper further situates this design against the atmospheric scattering model
\[
I(p) = J(p)e^{-\beta d(p)} + A(1-e^{-\beta d(p)}),
\]
arguing that paired residuals provide a practical conditional generative alternative to directly estimating all ASM factors [2508.11134].

## 4. Relation to bridge diffusion and acronym ambiguity

The RBDM label sits within a broader landscape of bridge-based diffusion methods, but not all bridge models are residual-shifting models, and not all uses of the acronym refer to the same idea.

| Model | Domain | Defining mechanism |
|---|---|---|
| ResShift [2403.07319] | SR, inpainting, blind face restoration | HQ-to-noisy-LQ residual shifting |
| RBDM [2508.11134] | Dehazing and haze generation | Dual residual-shifting Markov chains |
| Regulated Bridge Diffusion Model [2503.01352] | Polarization-based virtual staining | Brownian bridge with SSR and RR |
| RDBM [2510.23116] | Universal image restoration | Generalized OU bridge with residual-modulated diffusion |

The **Regulated Bridge Diffusion Model** for virtual staining defines a Brownian bridge between target modality \( x_0 \) and source modality \( y_0 \),
\[
q(x_t \mid x_0, y_0)
=
\mathcal{N}\bigl(x_t;\; (1-\tfrac{t}{T})x_0 + \tfrac{t}{T}y_0,\; \sigma_t I\bigr),
\quad
\sigma_t = \frac{2t(T-t)}{T^2}.
\]
To handle 16-channel Mueller matrix input, it introduces an encoder \( E_p \) with two convolutional layers, 16\(\to\)64 and 64\(\to\)3, plus Tanh, and supplements the bridge loss with **Starting State Regulation (SSR)** using a VGG-based perceptual loss and **Route Regulation (RR)** using MS-SSIM. Its complete objective is \( L = L_1 + L_2 + L_3 \) [2503.01352]. This model is bridge-based and bidirectional, but it is not presented as a residual-shifting efficient image restoration model.

The **Residual Diffusion Bridge Model** provides a different unification. It starts from a generalized OU process and sets the diffusion modulation variable to the residual
\[
\boldsymbol{\pi} = \mathbf{x}_0 - \boldsymbol{\mu},
\]
leading to the bridge SDE
\[
d\mathbf{x}_t
=
\theta_t\coth(\overline{\theta}_{t:T})(\boldsymbol{\mu}-\mathbf{x}_t)dt
+
\sqrt{2\boldsymbol{\pi}^2\lambda\theta_t}\,d\omega_t.
\]
Here the residual scales the diffusion term spatially, so intact regions receive little perturbation while degraded regions receive more aggressive reconstruction. The paper argues that Brownian bridges, VE bridges, VP bridges, OU bridges, stochastic interpolants, and even flow matching arise as special cases of this generalized bridge family [2510.23116]. A plausible implication is that the residual-shifting RBDM lineage and the analytically derived RDBM lineage can be read as complementary rather than competing: one emphasizes short conditional chains between paired endpoints, the other emphasizes a unified bridge theory with residual-modulated variance.

## 5. Architectures, objectives, and efficiency mechanisms

Architecturally, residual-based efficient bidirectional models are not tied to a single backbone. ResShift uses a U-shaped network with **Swin Transformer** blocks instead of global self-attention because global self-attention performs poorly when test resolution differs from training resolution, leading to uniform attention maps and blur; in some settings, diffusion is performed in the latent space of a pretrained VQGAN and then decoded back to images [2403.07319]. The dehazing RBDM uses a modified **UNet** derived from DDPM, with time embeddings for \( t_x \) and \( t_y \), and conditions jointly on noisy haze-free and hazy patches \( f_\theta(x_{t_x}^b, y_{t_y}^b, t_x, t_y) \) [2508.11134].

The training target also differs from standard DDPM conventions. In ResShift, the network predicts the clean HQ image \( \bm{x}_0 \), and the loss is an unweighted \( L_2 \) term augmented with LPIPS:
\[
\sum_t \big\| f_{\bm{\theta}}(\bm{x}_t,\bm{y}_0,t)-\bm{x}_0 \big\|_2^2
+
\lambda\, l_p\big(f_{\bm{\theta}}(\bm{x}_t,\bm{y}_0,t),\bm{x}_0\big).
\]
In the dehazing RBDM, the shared network is trained with an \( L_1 \) objective to recover the clean pair signal from jointly perturbed hazy and haze-free patches:
\[
\min_{\theta} \sum \omega \left\| f_\theta(x_{t_x}, y_{t_y}, t_x, t_y) - (x_0, y_0) \right\|_1,
\quad \omega=1.
\]
The virtual staining bridge model instead trains a noise predictor \( \epsilon_\theta(x_t,t,y_0) \) with SSR and RR as explicit regularizers [2403.07319, 2508.11134, 2503.01352].

Efficiency follows from the same structural principle across these variants: the trajectory is shortened because the forward process ends near the observed conditioning domain, not in isotropic noise. ResShift states this explicitly: the chain goes from HQ to noisy LQ, not to pure Gaussian noise, so only a few reverse steps are required. It further notes that methods such as StableSR and LDM train with 1000 steps and then rely on DDIM or DPM-Solver to reduce inference steps, with performance loss and oversmoothing, whereas ResShift avoids post-hoc acceleration and still restores with four steps [2403.07319]. The dehazing RBDM makes the same argument for bidirectional haze modeling, attributing its 15-step inference regime to residual-based forward processes, patch-based operation, and a shared model for both directions [2508.11134].

## 6. Empirical performance, limitations, and significance

The empirical record reported for residual-based efficient bidirectional models is task-specific but consistent in pattern: few-step residual-aware bridges remain competitive with, and often outperform, substantially longer diffusion pipelines.

For image restoration in ResShift, the paper reports that the method achieves superior or comparable performance on image super-resolution, image inpainting, and blind face restoration even only with four sampling steps. On ImageNet-Test SR, **ResShiftL-4** reports PSNR 25.02, SSIM 0.683, LPIPS 0.208, runtime 0.186 s, compared with **LDM-15** at PSNR 24.89, SSIM 0.670, LPIPS 0.269, runtime 0.247 s, and **StableSR-15** at PSNR 23.37, SSIM 0.631, LPIPS 0.262, runtime 1.070 s. An ablation further reports that \( T=4 \) already yields PSNR 25.64 and SSIM 0.6903 on ImageNet-Test SR, while increasing \( T \) beyond about 15 mainly improves LPIPS and is not necessary when perceptual loss is added [2403.07319].

For dehazing and haze generation, the explicit RBDM reports **PSNR = 32.68 dB** and **SSIM = 0.976** on RESIDE-6K, outperforming FFA-Net at 29.96/0.973 and TSNet at 31.31/0.975, while remaining second to DehazeFormer only in SSIM by a margin of 0.001. On NTIRE2020/2021/2023, it reports an average **PSNR = 23.89** and **SSIM = 0.7085**, compared with the best listed competitor, SCANet, at 20.37 and 0.6933. Its best ablation setting on NTIRE2020 is **15 steps, \( \kappa = 2.0 \), \( r = 32 \)**, with PSNR 23.19 and SSIM 0.6665 [2508.11134].

The bridge-based variants outside the strict residual-shifting definition show analogous gains. In virtual staining, full RBDM reports for polarization \(\to\) H&E: PSNR \(20.93 \pm 3.25\), SSIM \(0.53 \pm 0.18\), FID 19.67, LPIPS 0.26, improving over Pix2Pix and Pyramid Pix2Pix on all metrics; for polarization \(\to\) fluorescence it reports PSNR \(18.52 \pm 2.32\), SSIM \(0.44 \pm 0.12\), FID 43.03, LPIPS 0.39 [2503.01352]. RDBM reports state-of-the-art or best average PSNR/SSIM across deraining, low-light enhancement, desnowing, dehazing, and deblurring, with performance peaking at 10 sampling steps and with residual modulation outperforming settings such as \( \boldsymbol{\pi}=0 \) or \( \boldsymbol{\pi}=1 \) [2510.23116].

The limitations are equally consistent. ResShift notes that training degradations may not cover all real degradations and reports failures on some extremely degraded comic images. The dehazing RBDM reports performance drop under extremely heavy haze, where details are inherently absent, and notes that simple pixel-difference residuals may not capture all atmospheric effects. The virtual staining RBDM depends on a large, strictly registered paired dataset of 18,000 patches from 7 breast cancer patient samples, which constrains generalization to other organs or setups. RDBM, while analytically broader, still assumes paired HQ/LQ distributions and fixed bridge design choices such as the stationary variance parameter \( \lambda \) [2403.07319, 2508.11134, 2503.01352, 2510.23116].

Taken together, these papers establish RBDM not as a single immutable architecture but as a family resemblance among residual-guided bridge constructions. The recurring principle is that image restoration benefits when diffusion is reformulated as transport between paired domains, with residuals governing the direction and, in some formulations, even the spatial strength of the stochastic process. This suggests that the lasting significance of RBDM lies less in a specific acronym than in a design thesis: start from the observed degradation, build a short bridge to the target, and let residual structure determine how the bridge is traversed.

Source: https://www.emergentmind.com/topics/residual-based-efficient-bidirectional-diffusion-model-rbdm