Residual-Based Conditional Diffusion

• Residual-based conditional diffusion is a framework that models the residual error relative to a reference signal, enabling precise restoration and generation.
• It utilizes diverse conditioning mechanisms—such as direct concatenation, multi-path, and physics-guided sampling—to align diffusion dynamics with task-specific structures.
• Empirical studies in medical imaging, document enhancement, and time-series forecasting show improvements in fidelity, efficiency, and robustness over traditional methods.

Residual-based conditional diffusion denotes a family of diffusion frameworks in which generation, restoration, or probabilistic prediction is organized around a reference signal—such as a coarse predictor, a degraded observation, a prior model, or a previously reconstructed state—and the diffusion component models residual structure relative to that reference rather than the full target from scratch. Across recent work, the residual may be the primary diffusion variable, a drift term in the forward process, a conditioning signal recomputed during deterministic sampling, or a physics residual injected through the objective. This design appears in medical image segmentation, event-driven video reconstruction, document enhancement, probabilistic time-series forecasting, sparse-view CT, image dehazing, PDE surrogates, semiconductor TCAD surrogates, image restoration, PET/MR denoising, and robust conditional generation, with a conceptual precursor in residual-bridge methods for conditioned diffusions (Mao et al., 1 Sep 2025, Zhu et al., 2024, Yang et al., 2023, Lai et al., 2 Sep 2025, Choi et al., 3 Mar 2026, Liu et al., 15 Aug 2025, Park et al., 8 Jul 2025, Zhang et al., 28 Jun 2026, Zhang et al., 2023, Yoon et al., 2024, Xu et al., 2024, Malory et al., 2016).

1. Conceptual scope and historical lineage

Residual-based conditional diffusion is not a single algorithmic template. The literature instead shows a recurring modeling principle: a strong but imperfect estimate handles large-scale or low-frequency structure, while diffusion models the remaining uncertainty, detail, or correction. In "Prior-Guided Residual Diffusion: Calibrated and Efficient Medical Image Segmentation" (Mao et al., 1 Sep 2025), the prior is a deterministic nnU-Net-like segmentor; in "DocDiff: Document Enhancement via Residual Diffusion Models" (Yang et al., 2023), it is a coarse predictor $C_\theta$; in "Bridging Sequential Deep Operator Network and Video Diffusion: Residual Refinement of Spatio-Temporal PDE Solutions" (Park et al., 8 Jul 2025), it is an S-DeepONet video prior; and in "Residual Prior Diffusion: A Probabilistic Framework Integrating Coarse Latent Priors with Diffusion Models" (Kutsuna, 25 Dec 2025), it is a latent-variable Gaussian prior with decoder mean $\hat{\mu}(z)$.

A second lineage uses residuals to align the diffusion dynamics with the measurement process rather than with a pretrained predictor. "Temporal Residual Guided Diffusion Framework for Event-Driven Video Reconstruction" (Zhu et al., 2024) defines a temporal-domain residual $x_0^t = I^t - \tilde I^{t-1}$, where $\tilde I^{t-1}$ is a low-frequency estimate from the previous step. "Resfusion: Denoising Diffusion Probabilistic Models for Image Restoration Based on Prior Residual Noise" (Shi et al., 2023) inserts the residual $R=\hat{x}_0-x_0$ directly into the forward process. "Volumetric Conditional Score-based Residual Diffusion Model for PET/MR Denoising" (Yoon et al., 2024) models the volumetric residual $r=x_{\text{Low}}-x_{\text{Nor}}$ rather than PET intensities themselves.

A third lineage uses residuals as step-wise correction signals inside deterministic or physics-guided sampling. "ReCo-Diff: Residual-Conditioned Deterministic Sampling for Cold Diffusion in Sparse-View CT" (Choi et al., 3 Mar 2026) computes an observation residual $err_t=\mathcal{N}(x_t-D(\hat{x}_{0,t}^{\phi},v_t))$ at every step and conditions a second network pass on it. "PCGD: Physics-Guided Conditional Graph Diffusion for TCAD Device Simulation" (Zhang et al., 28 Jun 2026) does not diffuse a residual target, but it enforces quasi-Fermi gradient matching and noise-aware PDE residuals on $\hat{Y}_0$, so residuals enter as physics constraints rather than as the state variable.

The historical precursor is "Residual-Bridge Constructs for Conditioned Diffusions" (Malory et al., 2016), which applies modified diffusion bridge approximations to the difference between a true diffusion and an approximate diffusion driven by the same Brownian motion. This older bridge literature is not a DDPM literature, but it anticipates a central idea of later neural models: bridge or generate the discrepancy after extracting an approximate trajectory.

2. Residual variable and probabilistic parameterization

The literature instantiates the residual in several mathematically distinct locations.

Formulation	Residual definition	Representative papers
Target-space residual	$y-\text{prior}$, $x_{\rm gt}-x^C$, $\hat{\mu}(z)$0, $\hat{\mu}(z)$1, $\hat{\mu}(z)$2	(Mao et al., 1 Sep 2025, Yang et al., 2023, Zhu et al., 2024, Yoon et al., 2024, Park et al., 8 Jul 2025)
Prior-centered forward diffusion	Diffusion around $\hat{\mu}(z)$3 or $\hat{\mu}(z)$4 instead of around zero-mean Gaussian only	(Mao et al., 1 Sep 2025, Kutsuna, 25 Dec 2025)
Residual embedded in forward drift	$\hat{\mu}(z)$5 enters transition means; dual chains shift residuals between domains	(Shi et al., 2023, Liu et al., 15 Aug 2025)
Residual used during sampling	Observation residual computed from current estimate and measurements	(Choi et al., 3 Mar 2026)
Residual used in objectives	PDE residuals and gradient proxies act as physics regularizers	(Zhang et al., 28 Jun 2026)

In PGRD, the state is centered on a prior prediction $\hat{\mu}(z)$6, with residual $\hat{\mu}(z)$7 and clean residual $\hat{\mu}(z)$8. The forward process is

$\hat{\mu}(z)$9

which induces a standard Gaussian diffusion on $x_0^t = I^t - \tilde I^{t-1}$0. The denoiser predicts a residual-adapted velocity $x_0^t = I^t - \tilde I^{t-1}$1, and reconstruction is always $x_0^t = I^t - \tilde I^{t-1}$2 (Mao et al., 1 Sep 2025).

DocDiff uses the simpler residual

$x_0^t = I^t - \tilde I^{t-1}$3

then applies a standard DDPM forward process to $x_0^t = I^t - \tilde I^{t-1}$4. Its reverse process uses $x_0^t = I^t - \tilde I^{t-1}$5-prediction rather than $x_0^t = I^t - \tilde I^{t-1}$6-prediction, and the final image is $x_0^t = I^t - \tilde I^{t-1}$7 (Yang et al., 2023). RDIT similarly separates a point estimator $x_0^t = I^t - \tilde I^{t-1}$8 from a residual diffusion model over

$x_0^t = I^t - \tilde I^{t-1}$9

so that diffusion acts in a normalized residual space with approximately zero mean and unit variance (Lai et al., 2 Sep 2025).

Several works alter the forward law itself. Resfusion defines

$\tilde I^{t-1}$0

and modifies the Markov transition to

$\tilde I^{t-1}$1

then predicts a weighted residual noise $\tilde I^{t-1}$2 rather than pure $\tilde I^{t-1}$3 (Shi et al., 2023). RBDM for dehazing defines $\tilde I^{t-1}$4 and $\tilde I^{t-1}$5, then uses dual Markov chains that interpolate between haze-free and hazy domains through these residuals, allowing bidirectional translation with only $\tilde I^{t-1}$6 sampling steps (Liu et al., 15 Aug 2025). RPD moves still closer to a prior-conditioned probabilistic formalism: in prior-centered coordinates

$\tilde I^{t-1}$7

the forward process becomes standard DDPM diffusion on the residual deviation from the coarse latent prior (Kutsuna, 25 Dec 2025).

This suggests that “residual-based” can refer either to the target variable, the coordinate system, or the actual stochastic dynamics.

3. Conditioning mechanisms

Conditioning is as important as the residual definition. The surveyed methods show four recurring conditioning patterns.

The first is direct prior concatenation. DocDiff conditions the HRR denoiser $\tilde I^{t-1}$8 by channel-wise concatenation of the noisy residual and the coarse prediction $\tilde I^{t-1}$9 (Yang et al., 2023). The unified image restoration framework of "A Unified Conditional Framework for Diffusion-based Image Restoration" (Zhang et al., 2023) uses a lightweight UNet $R=\hat{x}_0-x_0$0 as spatial guidance and injects it into every diffusion block through a Conditional Integration Module and an Adaptive Kernel Guidance Module, so the diffusion model learns the residual $R=\hat{x}_0-x_0$1 under spatially adaptive conditioning. CSRD conditions residual denoising on low-dose PET, MR, and 3D patch coordinates, thereby learning $R=\hat{x}_0-x_0$2 rather than an unconditional residual distribution (Yoon et al., 2024).

The second is time-dependent multi-path conditioning. The event-driven video framework uses three conditioning paths: a pre-trained low-frequency intensity estimator $R=\hat{x}_0-x_0$3, a temporal recurrent encoder on event voxels $R=\hat{x}_0-x_0$4, and an attention-based high-frequency prior enhancement module that cross-attends between residual-event features, event features, and low-frequency features (Zhu et al., 2024). PGRD similarly conditions on both the image $R=\hat{x}_0-x_0$5 and the prior segmentation $R=\hat{x}_0-x_0$6 at every diffusion step, and explicitly distinguishes this from classifier guidance because the guidance is built into the forward and reverse chains rather than injected via gradients of an external classifier (Mao et al., 1 Sep 2025).

The third is global conditioning through structured tokens or operator priors. In PDE surrogates, video diffusion is conditioned on the S-DeepONet prior video by channel concatenation and on the loading history $R=\hat{x}_0-x_0$7 via FiLM (Park et al., 8 Jul 2025). PCGD injects boundary conditions and device structure into a MeshGraphNet denoiser via boundary tokens, structure tokens, and global cross-attention from mesh nodes to condition tokens (Zhang et al., 28 Jun 2026). RDIT concatenates time-series history $R=\hat{x}_0-x_0$8, point forecasts $R=\hat{x}_0-x_0$9, residual states $r=x_{\text{Low}}-x_{\text{Nor}}$0, and diffusion-step embeddings, then processes them with a bidirectional Mamba network (Lai et al., 2 Sep 2025).

The fourth is conditioning through corrective residuals computed during inference. ReCo-Diff first predicts a null reconstruction $r=x_{\text{Low}}-x_{\text{Nor}}$1, re-degrades it to the current view level, forms the residual $r=x_{\text{Low}}-x_{\text{Nor}}$2, and then performs a second conditioned network call $r=x_{\text{Low}}-x_{\text{Nor}}$3 (Choi et al., 3 Mar 2026). RCDM uses a different residual notion: the robust guidance term is

$r=x_{\text{Low}}-x_{\text{Nor}}$4

so the residual between conditional and unconditional denoisers becomes the adjustable guidance signal (Xu et al., 2024).

4. Objectives, parameterizations, and samplers

Residual-based conditional diffusion does not imply a single training objective. The surveyed methods use $r=x_{\text{Low}}-x_{\text{Nor}}$5-prediction, $r=x_{\text{Low}}-x_{\text{Nor}}$6-prediction, $r=x_{\text{Low}}-x_{\text{Nor}}$7-prediction, direct denoising, auxiliary supervision, and physics regularization.

PGRD adopts a residual-adapted velocity parameterization with

$r=x_{\text{Low}}-x_{\text{Nor}}$8

and trains with an MSE loss on $r=x_{\text{Low}}-x_{\text{Nor}}$9, augmented by Deep Diffusion Supervision: $err_t=\mathcal{N}(x_t-D(\hat{x}_{0,t}^{\phi},v_t))$0 DDS attaches decoder heads at selected timesteps and applies voxel-wise cross-entropy to intermediate predictions (Mao et al., 1 Sep 2025).

DocDiff instead uses $err_t=\mathcal{N}(x_t-D(\hat{x}_{0,t}^{\phi},v_t))$1-prediction for the residual and a deterministic DDIM-style sampler with variance set to zero. It augments the diffusion loss with frequency separation: a low-frequency loss for the coarse predictor and a high-frequency loss for the residual diffusion model, with $err_t=\mathcal{N}(x_t-D(\hat{x}_{0,t}^{\phi},v_t))$2 and $err_t=\mathcal{N}(x_t-D(\hat{x}_{0,t}^{\phi},v_t))$3 (Yang et al., 2023). Event-driven reconstruction uses standard DDPM $err_t=\mathcal{N}(x_t-D(\hat{x}_{0,t}^{\phi},v_t))$4-prediction with an $err_t=\mathcal{N}(x_t-D(\hat{x}_{0,t}^{\phi},v_t))$5 loss on the residual image, and recurrent temporal state updates via ConvLSTM (Zhu et al., 2024).

RPD proves that optimization reduces to familiar noise-prediction or velocity-prediction objectives in prior-centered coordinates, and introduces auxiliary variables such as

$err_t=\mathcal{N}(x_t-D(\hat{x}_{0,t}^{\phi},v_t))$6

to reduce prediction difficulty when the prior reconstruction is accurate (Kutsuna, 25 Dec 2025). Resfusion also changes the prediction target, replacing $err_t=\mathcal{N}(x_t-D(\hat{x}_{0,t}^{\phi},v_t))$7 with

$err_t=\mathcal{N}(x_t-D(\hat{x}_{0,t}^{\phi},v_t))$8

while preserving the DDPM reverse form (Shi et al., 2023).

Physics-guided variants use hybrid objectives. PCGD adds exponent-free quasi-Fermi gradient matching

$err_t=\mathcal{N}(x_t-D(\hat{x}_{0,t}^{\phi},v_t))$9

and a noise-aware PDE residual loss

$\hat{Y}_0$0

combined as

$\hat{Y}_0$1

with $\hat{Y}_0$2, $\hat{Y}_0$3, $\hat{Y}_0$4, and $\hat{Y}_0$5 (Zhang et al., 28 Jun 2026).

Sampling strategies are equally diverse. PGRD uses DDIM-style sampling with $\hat{Y}_0$6 and reports near-peak Dice at about $\hat{Y}_0$7 steps, whereas vanilla DDPM required more than $\hat{Y}_0$8 steps (Mao et al., 1 Sep 2025). DocDiff emphasizes deterministic few-step sampling and reports strong perceptual quality even with $\hat{Y}_0$9 steps (Yang et al., 2023). ReCo-Diff keeps a deterministic cold-diffusion update but adds residual-conditioned self-guidance at each step (Choi et al., 3 Mar 2026). The image restoration framework introduces inter-step patch-splitting for arbitrary-resolution inference without grid artifacts (Zhang et al., 2023).

5. Reported domains and empirical behavior

The empirical record is heterogeneous but consistent in one respect: most papers report that residual-based conditionalization improves either fidelity, calibration, efficiency, or robustness relative to direct diffusion baselines.

In medical segmentation, PGRD reports higher Dice and lower NLL/ECE than Bayesian, ensemble, Probabilistic U-Net, and vanilla diffusion baselines, while requiring fewer sampling steps. On BraTS2024, DDPM achieved DSC $y-\text{prior}$0, NLL $y-\text{prior}$1, ECE $y-\text{prior}$2, whereas PGRD achieved DSC $y-\text{prior}$3, NLL $y-\text{prior}$4, ECE $y-\text{prior}$5. On INSTANCE2022, DDPM achieved DSC $y-\text{prior}$6, NLL $y-\text{prior}$7, ECE $y-\text{prior}$8, whereas PGRD achieved DSC $y-\text{prior}$9, NLL $x_{\rm gt}-x^C$0, ECE $x_{\rm gt}-x^C$1 (Mao et al., 1 Sep 2025).

In document enhancement, DocDiff reports that DocDiff (Non-native)-5, with $x_{\rm gt}-x^C$2M parameters, achieved MANIQA $x_{\rm gt}-x^C$3, LPIPS $x_{\rm gt}-x^C$4, PSNR $x_{\rm gt}-x^C$5, and SSIM $x_{\rm gt}-x^C$6 on deblurring, compared with MPRNet’s MANIQA $x_{\rm gt}-x^C$7, LPIPS $x_{\rm gt}-x^C$8, PSNR $x_{\rm gt}-x^C$9, and SSIM $\hat{\mu}(z)$00. It also reports OCR gains, with character error rate decreasing from about $\hat{\mu}(z)$01 after DE-GAN to about $\hat{\mu}(z)$02 after DE-GAN+HRR and about $\hat{\mu}(z)$03 after DocDiff (Yang et al., 2023).

In probabilistic time-series forecasting, RDIT is evaluated on Traffic, Weather, Electricity, Exchange, Solar, ETTh1, ETTm1, and ETTm2, and the paper states that it achieves the best overall CRPS on almost all datasets, the smallest or near-smallest PICP distance on most datasets, and state-of-the-art or second-best MAE and MSE among PTSF models when distributions are collapsed to means (Lai et al., 2 Sep 2025). This suggests that residual diffusion can improve probabilistic calibration without sacrificing point accuracy when a strong point forecaster is available.

In sparse-view CT, ReCo-Diff reports quantitative gains over cold-diffusion baselines. For $\hat{\mu}(z)$04 views, CvG-Diff achieved RMSE $\hat{\mu}(z)$05, PSNR $\hat{\mu}(z)$06 dB, SSIM $\hat{\mu}(z)$07, whereas ReCo-Diff achieved RMSE $\hat{\mu}(z)$08, PSNR $\hat{\mu}(z)$09 dB, SSIM $\hat{\mu}(z)$10. For $\hat{\mu}(z)$11 views, CvG-Diff achieved RMSE $\hat{\mu}(z)$12, PSNR $\hat{\mu}(z)$13 dB, SSIM $\hat{\mu}(z)$14, whereas ReCo-Diff achieved RMSE $\hat{\mu}(z)$15, PSNR $\hat{\mu}(z)$16 dB, SSIM $\hat{\mu}(z)$17 (Choi et al., 3 Mar 2026).

In dehazing, RBDM reports PSNR $\hat{\mu}(z)$18 and SSIM $\hat{\mu}(z)$19 on RESIDE-6K, and on NTIRE 2020 it reports $\hat{\mu}(z)$20, versus SCANet’s $\hat{\mu}(z)$21. It further reports that $\hat{\mu}(z)$22 steps outperform both $\hat{\mu}(z)$23 and $\hat{\mu}(z)$24 steps on NTIRE2020, with $\hat{\mu}(z)$25 at $\hat{\mu}(z)$26 steps compared with $\hat{\mu}(z)$27 at $\hat{\mu}(z)$28 and $\hat{\mu}(z)$29 at $\hat{\mu}(z)$30 (Liu et al., 15 Aug 2025).

In PDE surrogates, the residual video-diffusion model conditioned on S-DeepONet reduces the mean relative $\hat{\mu}(z)$31 error from $\hat{\mu}(z)$32 to $\hat{\mu}(z)$33 on the cavity-flow benchmark and from $\hat{\mu}(z)$34 to $\hat{\mu}(z)$35 on the dogbone plasticity benchmark (Park et al., 8 Jul 2025). In TCAD surrogates, PCGD reports that deterministic direct regression reaches mean relative field error $\hat{\mu}(z)$36, local diffusion reaches $\hat{\mu}(z)$37, pure condition-aware diffusion reaches $\hat{\mu}(z)$38, and full PCGD reaches $\hat{\mu}(z)$39 while reducing the aggregate residual metric from $\hat{\mu}(z)$40 to $\hat{\mu}(z)$41 (Zhang et al., 28 Jun 2026).

6. Misconceptions, limitations, and open directions

A common misconception is that residual-based conditional diffusion is equivalent to classifier guidance or to ordinary conditional concatenation. The surveyed work shows otherwise. PGRD explicitly distinguishes segmentor-as-prior guidance from classifier or score guidance, because the prior enters the forward and reverse chains directly rather than through gradients of a separate classifier (Mao et al., 1 Sep 2025). ReCo-Diff likewise contrasts residual-conditioned self-guided sampling with classifier guidance and classifier-free guidance: the guidance signal is the observation residual in image space, not a score-space interpolation (Choi et al., 3 Mar 2026).

Another misconception is that residualization is universally beneficial. The AR diffusion study on conditional dependence shows that structured factorization helps only when the partition and ordering align with the true dependency structure; when patches break the natural dependency, AR diffusion can underperform vanilla DDPM (Huang et al., 30 Apr 2025). This suggests that residual targets and conditional factorizations must match the problem geometry or physics.

Several limitations recur. Residual methods often depend on a strong coarse prior. RPD states that if the prior $\hat{\mu}(z)$42 is poor, few-step benefits may disappear and auxiliary variables lose effectiveness (Kutsuna, 25 Dec 2025). The PDE surrogate paper notes that diffusion remains relatively slow versus S-DeepONet alone, even though it is far faster than CFD or FEA (Park et al., 8 Jul 2025). ReCo-Diff and the event-driven video framework both retain long iterative samplers in their strongest settings, with $\hat{\mu}(z)$43 or more steps for sparse-view CT and $\hat{\mu}(z)$44 diffusion steps for event-driven reconstruction during analysis (Choi et al., 3 Mar 2026, Zhu et al., 2024). PCGD reports poor zero-shot transfer to completely new SOI topologies before LoRA adaptation, indicating that residual-based physics-aware diffusion is not automatically topology invariant (Zhang et al., 28 Jun 2026). CSRD depends on 3D patch-wise training and MR conditioning to maintain volumetric coherence; without MR, residual noise remains more visible in background regions (Yoon et al., 2024).

Open directions arise directly from these limits. RPD proposes multi-level hierarchical priors and latent-space variants (Kutsuna, 25 Dec 2025). The PDE surrogate work points to diffusion distillation, mixture-of-experts operator priors, and stronger physics-informed diffusion objectives (Park et al., 8 Jul 2025). PCGD identifies PDE-guided sampling, broader pretraining, and 3D scaling as natural next steps (Zhang et al., 28 Jun 2026). More broadly, the surveyed literature suggests that residual-based conditional diffusion is best understood as a modular pattern: select an informative reference, define a residual whose statistics are simpler than those of the full target, choose a conditioning mechanism that preserves the relevant dependencies, and align the sampler or loss with the structure of the residual itself.