Residual Conditional Diffusion Models

• Residual conditional diffusion models are a class of generative methods that use condition-dependent priors to model only the remaining discrepancy from coarse predictions.
• They employ diverse techniques such as explicit residual extraction, residual trajectory shifting, and prior-centered normalization to tailor the diffusion process for tasks like medical imaging and video editing.
• Advanced training strategies, including staged training and intermediate supervision, stabilize the diffusion process and improve reconstruction quality, outperforming conventional methods in several benchmarks.

Residual conditional diffusion model denotes a class of diffusion-based generative models in which denoising is conditioned on auxiliary information and organized around a residual quantity, or around a diffusion trajectory shifted toward a condition-dependent baseline, rather than around unconditional synthesis from a simple origin alone. In recent formulations, the baseline has taken the form of a coarse prior segmentation, a low-dose medical image, a radar observation, a previous edited frame, a previous intensity estimate, a latent prior sample, or a coarse neural-operator prediction; correspondingly, the learned stochastic object may be an explicit residual field, a prior-centered normalized latent, or a reverse process whose forward geometry follows an observation residual path (Mao et al., 1 Sep 2025, Yoon et al., 2024, Wang et al., 21 Mar 2026, Zhang et al., 2023, Salehi et al., 6 Feb 2026, Kutsuna, 25 Dec 2025, Park et al., 8 Jul 2025, Zhu et al., 2024).

1. Conceptual scope

A residual conditional diffusion model is not a single canonical parameterization. Across the literature, the common pattern is that a condition-dependent reference is supplied first, and diffusion is used to model the remaining discrepancy or a trajectory anchored to that reference. In medical segmentation, the reference is a coarse prior mask; in PET/MR denoising it is the low-dose PET together with MR anatomy; in event-driven reconstruction it is a previous intensity estimate; in causal video editing it is the previous edited frame; in PDE surrogates it is a coarse S-DeepONet prediction; and in latent-prior hybrids it is the decoder output of a pretrained prior model (Mao et al., 1 Sep 2025, Yoon et al., 2024, Zhu et al., 2024, Salehi et al., 6 Feb 2026, Park et al., 8 Jul 2025, Kutsuna, 25 Dec 2025).

This diversity is visible in the object being diffused. Some models diffuse an explicit residual, such as $$\mathbf r_0 = \mathbf y_\star - \pi_\phi(\mathbf X)$$ in prior-guided segmentation, $r = x_{\text{Low}} - x_{\text{Nor}}$ in PET denoising, $x^t_0 = I^t - \tilde I^{t-1}$ in event reconstruction, or $$\mathcal R = \mathbf{x}^{GT} - \mathbf{x}^{\text{prior}}$$ in PDE residual refinement (Mao et al., 1 Sep 2025, Yoon et al., 2024, Zhu et al., 2024, Park et al., 8 Jul 2025). Others instead define a residual path or prior-centered coordinates, as in observation-anchored mmWave respiration sensing and Residual Prior Diffusion (Wang et al., 21 Mar 2026, Kutsuna, 25 Dec 2025).

Representative formulation	Residual or shifted object	Conditioning source
PGRD (Mao et al., 1 Sep 2025)	$$\mathbf r_0 = \mathbf y_\star - \pi_\phi(\mathbf X)$$	image $\mathbf X$ and prior $\pi_\phi(\mathbf X)$
CSRD (Yoon et al., 2024)	$r = x_{\text{Low}} - x_{\text{Nor}}$	low-dose PET, MR, patch coordinates
mmWave-Diffusion (Wang et al., 21 Mar 2026)	residual path $z = y - x$	radar phase observation $y$
UCDIR (Zhang et al., 2023)	$r = x_{\text{Low}} - x_{\text{Nor}}$0	degraded image, guidance image, metadata
RFDM (Salehi et al., 6 Feb 2026)	$r = x_{\text{Low}} - x_{\text{Nor}}$1	source frame, prompt, previous output
TRDF (Zhu et al., 2024)	$r = x_{\text{Low}} - x_{\text{Nor}}$2	event voxel, previous estimate, recurrent state
RPD (Kutsuna, 25 Dec 2025)	prior-centered $r = x_{\text{Low}} - x_{\text{Nor}}$3	latent prior $r = x_{\text{Low}} - x_{\text{Nor}}$4

This suggests that the term is best treated as a design pattern rather than as a single algorithmic template.

2. Mathematical archetypes

One archetype is explicit residual diffusion around a prior. In Prior-Guided Residual Diffusion, the target segmentation is rewritten as a one-hot field $r = x_{\text{Low}} - x_{\text{Nor}}$5, a coarse predictor produces $r = x_{\text{Low}} - x_{\text{Nor}}$6, and the residual target is defined by

$r = x_{\text{Low}} - x_{\text{Nor}}$7

The forward process includes prior drift,

$r = x_{\text{Low}} - x_{\text{Nor}}$8

so that in centered coordinates the residual follows the standard diffusion form

$r = x_{\text{Low}} - x_{\text{Nor}}$9

The reverse chain is then explicitly guided by both image and prior (Mao et al., 1 Sep 2025).

A second archetype is score-based residual diffusion. In CSRD, the residual is

$x^t_0 = I^t - \tilde I^{t-1}$0

and the reverse dynamics are defined through the EDM probability-flow ODE

$x^t_0 = I^t - \tilde I^{t-1}$1

The denoiser $x^t_0 = I^t - \tilde I^{t-1}$2 predicts the clean residual from a noisy residual, while the score is recovered as

$x^t_0 = I^t - \tilde I^{t-1}$3

Here the residual is the stochastic variable itself, and the conditioning appears in the score field over residual space (Yoon et al., 2024).

A third archetype alters the forward geometry rather than only redefining the target. In mmWave-Diffusion, the residual

$x^t_0 = I^t - \tilde I^{t-1}$4

defines the forward Markov chain

$x^t_0 = I^t - \tilde I^{t-1}$5

with marginal

$x^t_0 = I^t - \tilde I^{t-1}$6

Since $x^t_0 = I^t - \tilde I^{t-1}$7, the mean interpolates from clean respiration toward the observation rather than toward an unconditional Gaussian reference. Reverse sampling is initialized in an Observation-Consistent Neighborhood,

$x^t_0 = I^t - \tilde I^{t-1}$8

which keeps the reverse chain close to the measurement manifold (Wang et al., 21 Mar 2026).

A fourth archetype is prior-centered normalization. In Residual Prior Diffusion,

$x^t_0 = I^t - \tilde I^{t-1}$9

and the forward process becomes

$$\mathcal R = \mathbf{x}^{GT} - \mathbf{x}^{\text{prior}}$$0

In these coordinates, diffusion models the normalized deviation from the coarse latent prior rather than the raw data distribution (Kutsuna, 25 Dec 2025).

3. Conditioning pathways and prediction parameterizations

Residual conditional diffusion does not imply a unique network output. In PGRD, the denoiser does not predict the clean segmentation, the raw Gaussian noise, or the residual directly; it predicts a residual-centered $$\mathcal R = \mathbf{x}^{GT} - \mathbf{x}^{\text{prior}}$$1-parameterization,

$$\mathcal R = \mathbf{x}^{GT} - \mathbf{x}^{\text{prior}}$$2

from which the clean residual and clean segmentation are reconstructed (Mao et al., 1 Sep 2025). In CSRD, by contrast, the denoiser predicts the clean residual and the score is then recovered from the EDM identity (Yoon et al., 2024). In UCDIR and TRDF, the diffusion network is trained in $$\mathcal R = \mathbf{x}^{GT} - \mathbf{x}^{\text{prior}}$$3-prediction mode on a residual target rather than on the full image (Zhang et al., 2023, Zhu et al., 2024). In mmWave-Diffusion, the process is residual-defined but the reverse model predicts the clean respiratory waveform $$\mathcal R = \mathbf{x}^{GT} - \mathbf{x}^{\text{prior}}$$4, not the residual $$\mathcal R = \mathbf{x}^{GT} - \mathbf{x}^{\text{prior}}$$5 (Wang et al., 21 Mar 2026). In RFDM, the residual-flow forward process is temporal, yet the denoiser is trained in $$\mathcal R = \mathbf{x}^{GT} - \mathbf{x}^{\text{prior}}$$6-prediction form on the full edited frame $$\mathcal R = \mathbf{x}^{GT} - \mathbf{x}^{\text{prior}}$$7 (Salehi et al., 6 Feb 2026).

Conditioning is likewise heterogeneous. Some methods inject conditions architecturally at every denoising step. PGRD conditions on $$\mathcal R = \mathbf{x}^{GT} - \mathbf{x}^{\text{prior}}$$8, with the prior entering the forward process, the denoiser input, and the reconstruction path (Mao et al., 1 Sep 2025). UCDIR combines degraded input, guidance image, timestep, and scalar metadata through a Conditional Integration Module and Adaptive Kernel Guidance Module, yielding spatially adaptive dynamic kernels

$$\mathcal R = \mathbf{x}^{GT} - \mathbf{x}^{\text{prior}}$$9

inside every diffusion block (Zhang et al., 2023). TRDF uses three conditioning streams—previous intensity estimate, recurrent event features, and a noisy-residual/event branch—and fuses them by cross-encoder attention (Zhu et al., 2024). RFDM conditions each frame on the current source frame, previous edited output, prompt, and noise level, with the prompt entering by cross-attention and the frame pair by concatenation (Salehi et al., 6 Feb 2026).

A separate lineage modifies the diffusion trajectory itself. ShiftDDPMs adds a condition-dependent offset

$$\mathbf r_0 = \mathbf y_\star - \pi_\phi(\mathbf X)$$0

so that the condition shifts the latent path at every timestep, not only the reverse denoiser (Zhang et al., 2023). This is adjacent to residual conditional diffusion, though the paper frames it as trajectory shifting rather than residual modeling.

4. Architectural patterns and training strategies

A recurrent pattern is staged training around a frozen or pretrained prior. PGRD trains the prior network first and then freezes it while training the diffusion model (Mao et al., 1 Sep 2025). RPD assumes a pretrained latent-variable prior and then trains diffusion on the prior-centered residual coordinates (Kutsuna, 25 Dec 2025). In the PDE hybrid surrogate, S-DeepONet is trained first, frozen, and its coarse output becomes the conditioning prior for residual video diffusion (Park et al., 8 Jul 2025). TRDF likewise relies on a pre-trained low-frequency intensity estimator and then learns diffusion over the temporal residual (Zhu et al., 2024). This staged structure reflects the division of labor between coarse deterministic structure and stochastic refinement.

Training objectives are equally task-specific. PGRD uses a residual-velocity loss together with Deep Diffusion Supervision, a selected-step auxiliary segmentation loss that stabilizes intermediate denoising states (Mao et al., 1 Sep 2025). CSRD uses patch-wise residual reconstruction MSE in an EDM denoising-score-matching formulation over $$\mathbf r_0 = \mathbf y_\star - \pi_\phi(\mathbf X)$$1 volumetric patches (Yoon et al., 2024). RFDM addresses exposure bias by using diffusion forcing during autoregressive video training rather than pure teacher forcing (Salehi et al., 6 Feb 2026). The PDE residual refiner applies a time-wise focal loss that reweights framewise reconstruction errors across the video clip (Park et al., 8 Jul 2025). UCDIR introduces inter-step patch-splitting to support arbitrary-resolution image restoration without grid artifacts (Zhang et al., 2023).

This suggests that residual conditional diffusion is often coupled with explicit optimization aids that are specific to the residual structure being modeled: intermediate supervision for segmentation, patch-wise decomposition for volumetric denoising, recurrent rollout strategies for causal video, and temporal reweighting for space-time fields.

5. Applications and empirical behavior

Medical image segmentation provides one of the clearest demonstrations of the paradigm. On BraTS2024 and INSTANCE2022, PGRD reports DSC $$\mathbf r_0 = \mathbf y_\star - \pi_\phi(\mathbf X)$$2, NLL $$\mathbf r_0 = \mathbf y_\star - \pi_\phi(\mathbf X)$$3, ECE $$\mathbf r_0 = \mathbf y_\star - \pi_\phi(\mathbf X)$$4 on BraTS2024 and DSC $$\mathbf r_0 = \mathbf y_\star - \pi_\phi(\mathbf X)$$5, NLL $$\mathbf r_0 = \mathbf y_\star - \pi_\phi(\mathbf X)$$6, ECE $$\mathbf r_0 = \mathbf y_\star - \pi_\phi(\mathbf X)$$7 on INSTANCE2022, while its ablation without prior-guided residual learning degrades both Dice and calibration. The same paper reports that PGRD reaches near-peak Dice at around 300 steps, whereas vanilla DDPM requires over 800 steps (Mao et al., 1 Sep 2025).

In PET/MR denoising, CSRD models only the PET residual and uses 3D patch-wise conditional score modeling. For a full $$\mathbf r_0 = \mathbf y_\star - \pi_\phi(\mathbf X)$$8 PET volume, the paper reports about 3 minutes denoising time, about 12 GB GPU memory, and 100 function evaluations at inference. Quantitatively, CSRD with MR achieves MAE $$\mathbf r_0 = \mathbf y_\star - \pi_\phi(\mathbf X)$$9, PSNR $\mathbf X$0, SSIM $\mathbf X$1, $\mathbf X$2 $\mathbf X$3, and $\mathbf X$4 $\mathbf X$5 (Yoon et al., 2024).

Measurement-anchored and temporal variants emphasize inference efficiency. mmWave-Diffusion uses $\mathbf X$6 forward steps but only 20 reverse steps, reporting waveform-reconstruction CS $\mathbf X$7, MSE $\mathbf X$8, and respiratory-rate MAE $\mathbf X$9 BPM on 13.25 hours of synchronized radar-respiration data (Wang et al., 21 Mar 2026). RFDM performs causal video editing frame by frame, with 16-frame latency of 8 s and 2 GB RAM for RFDM1.5 and 13 s and 6 GB RAM for RFDM3.5 on an A100, while its residual-flow formulation improves error accumulation and tracking metrics relative to frame prediction (Salehi et al., 6 Feb 2026). RBDM, for bidirectional dehazing and haze generation, reports effective size-agnostic transitions with only 15 sampling steps and PSNR $\pi_\phi(\mathbf X)$0, SSIM $\pi_\phi(\mathbf X)$1 at that operating point (Liu et al., 15 Aug 2025).

Residual refinement also appears effective when a strong deterministic prior already captures low-frequency structure. In PDE surrogates, conditioning video diffusion on an S-DeepONet prior and predicting only the residual lowers the mean relative $\pi_\phi(\mathbf X)$2 error from $\pi_\phi(\mathbf X)$3 to $\pi_\phi(\mathbf X)$4 for lid-driven cavity flow and from $\pi_\phi(\mathbf X)$5 to $\pi_\phi(\mathbf X)$6 for tensile plasticity (Park et al., 8 Jul 2025). In event-driven video reconstruction, TRDF reports SSIM improvements over the second-best method of $\pi_\phi(\mathbf X)$7 on IJRR, $\pi_\phi(\mathbf X)$8 on HQF, and $\pi_\phi(\mathbf X)$9 on MVSEC, which the paper attributes to combining temporal accumulation, low-frequency priors, and high-frequency residual guidance (Zhu et al., 2024).

6. Conceptual distinctions and common misconceptions

A common misconception is that “residual conditional diffusion” means the network must output a residual tensor directly. The literature does not support that simplification. PGRD predicts residual-centered $r = x_{\text{Low}} - x_{\text{Nor}}$0, CSRD predicts a clean residual, TRDF predicts $r = x_{\text{Low}} - x_{\text{Nor}}$1 over a residual target, mmWave-Diffusion predicts the clean signal $r = x_{\text{Low}} - x_{\text{Nor}}$2 while using a residual-defined process, and RFDM predicts the clean edited frame despite a residualized forward path (Mao et al., 1 Sep 2025, Yoon et al., 2024, Zhu et al., 2024, Wang et al., 21 Mar 2026, Salehi et al., 6 Feb 2026).

A second misconception is that any conditional diffusion model with residual blocks is thereby a residual conditional diffusion model. The CBCT-to-CT DDPM uses a time-embedded U-Net with residual blocks and attention blocks, but the paper does not formulate CT synthesis in residual image space; the residuality is architectural, not a residual diffusion target (Peng et al., 2023). Conversely, some models are residual conditional in process design even when the denoiser output is not a residual variable.

A third misconception concerns terminology. “RCDM” in “RCDM: Enabling Robustness for Conditional Diffusion Model” denotes Robust Conditional Diffusion Model, not Residual Conditional Diffusion Model; its contribution is dynamic sampling-time reweighting of conditional and unconditional predictors for robustness, not residual-target diffusion (Xu et al., 2024). “Residual Context Diffusion” in diffusion LLMs is a cross-step residual-conditioning mechanism over discarded token distributions, not a vision-style residual diffusion process (Hu et al., 30 Jan 2026).

There are also adjacent but non-identical formulations. “Target Speech Extraction with Conditional Diffusion Model” is explicitly described as a conditional score-based diffusion model, not an explicit residual conditional diffusion model, even though its mixture-centered drift admits a residual interpretation (Kamo et al., 2023). “On conditional diffusion models for PDE simulations” develops conditional, guided, and hybrid score models for forecasting and data assimilation, but does not introduce an explicit residual state formulation (Shysheya et al., 2024). ShiftDDPMs introduces condition-dependent trajectory shifts, which can be interpreted as additive residual offsets in latent space, but the paper frames the method as shifting diffusion trajectories rather than as residual diffusion proper (Zhang et al., 2023).

Taken together, these distinctions suggest that residual conditional diffusion is best understood as a family of designs in which conditional generation is narrowed from full-signal synthesis to correction, refinement, or condition-anchored transport. What varies from paper to paper is where the residual enters: in the target variable, the latent coordinates, the forward drift, the reverse initialization, the conditioning pathway, or the posterior score correction.