Deep Diffusion Image Prior (DDIP)

• Deep Diffusion Image Prior (DDIP) is a framework that adapts pre-trained diffusion priors with test-time optimization to tackle out-of-distribution inverse problems.
• It leverages a multi-scale adaptation strategy inspired by Deep Image Prior (DIP) to integrate generative statistics with measurement fidelity.
• Variants like D3IP and uDiG-DIP demonstrate improved reconstruction accuracy and efficiency across applications in 3D imaging, MRI, CT, PET, and seismic data.

Searching arXiv for papers on Deep Diffusion Image Prior and closely related variants. Deep Diffusion Image Prior (DDIP) denotes a class of inverse-problem methods that combine a diffusion prior with test-time, image-specific adaptation in the spirit of Deep Image Prior (DIP). In the strict formulation introduced for out-of-distribution (OOD) inverse problems, DDIP treats diffusion-based posterior sampling as a multi-scale DIP over the reverse diffusion trajectory, adapting the prior at each timestep by minimizing a measurement-fidelity objective; in broader later usage, the term also covers hybrids in which a pre-trained diffusion model is coupled to a DIP-style data-consistency module, adaptive input refinement, or online fine-tuning against the test measurement (Chung et al., 2024). DDIP is therefore neither synonymous with classical DIP nor with diffusion-based inverse solvers in general, and it must also be distinguished from the unrelated acronym usage “double deep image prior” in multislice ptychography (Du et al., 2021).

1. Definition and conceptual scope

DDIP addresses a recurrent failure mode of generative diffusion priors in inverse problems: the training distribution and the test distribution may differ substantially. The formal setting is

$y = Ax + n,\qquad n\sim \mathcal N(0,\sigma_y^2I),$

with posterior sampling from $p(x\mid y)\propto p_{\rm data}(x)p(y\mid x)$. The central claim of the DDIP framework is that a fixed diffusion prior is often inadequate under OOD conditions, and that the prior should instead be adapted during sampling using only the measurement available at test time (Chung et al., 2024).

Within this scope, DDIP has two levels of meaning. In the narrow sense, it refers to the framework of timestep-wise diffusion-prior adaptation developed for 3D inverse problems, together with its accelerated 3D variant D3IP (Chung et al., 2024). In a broader operational sense, later works use the DDIP idea to describe alternating diffusion sampling and model fine-tuning for PET reconstruction, diffusion-guided DIP reconstruction for MRI and CT, and constrained diffusion-driven seismic interpolation, all of which preserve the core principle that the image prior is not kept fixed but is adapted or coupled to the specific test instance (Hashimoto et al., 20 Jul 2025, Liang et al., 2024, Goyes-Peñafiel et al., 2024).

This usage implies a family resemblance rather than a single algorithm. The family is unified by three properties: a pre-trained diffusion prior, a test-time optimization loop tied to the observed measurement, and an image-specific inductive bias reminiscent of DIP rather than a purely feed-forward supervised regressor. This suggests that DDIP is best understood as a framework for scan-adaptive or measurement-adaptive regularization built on diffusion models.

2. From Deep Image Prior to diffusion-time adaptation

The immediate intellectual precursor of DDIP is DIP. In the inverse-problem formulation analyzed in “Regularization by architecture: A deep prior approach for inverse problems” (Dittmer et al., 2018), DIP reconstructs by optimizing network parameters for a single datum: $\min_\Theta \|A\varphi_\Theta(z)-y^\delta\|^2.$ That work formalizes “regularization by architecture”: the network architecture itself restricts the attainable solution set, and for specific unrolled proximal architectures the method can be reinterpreted as optimization over Tikhonov functionals. The same paper also connects trivial DIP parameterizations to Landweber iteration and derives a “Soft TSVD” spectral filter as an order-optimal regularization method (Dittmer et al., 2018).

DDIP generalizes this idea from a single optimization over a fixed network output to repeated optimization along the diffusion probability-flow ODE. For the unconditional diffusion prior,

$$dx_t = -t\nabla_{x_t}\log p(x_t)\,dt = \frac{x_t - [x_0|x_t]}{t}\,dt,$$

with Tweedie’s formula

$[x_0|x_t] = x_t + t^2\nabla_{x_t}\log p(x_t).$

The conditional version replaces $[x_0|x_t]$ by $[x_0|x_t,y]$. DDIP observes that if a diffusion inverse solver provides an estimate $D_\theta(x_t|y)\approx [x_0|x_t,y]$, then one can adapt $\theta$ at each timestep by minimizing a DIP-style fidelity loss. The canonical recursion is

$\text{for } t=T,\ldots,1:\quad \begin{cases} \theta_{t-1}\gets \argmin_{\theta_t}\|y - D_{\theta_t}(x_t|y)\|_2^2,\[2mm] x_{t-1}\gets \mathcal S_{\theta_{t-1}}\!\left(D_{\theta_{t-1}}(x_t|y),\eta\right). \end{cases}$

The framework is explicitly described as a “multi-scale DIP” over the diffusion trajectory (Chung et al., 2024).

This connection matters because it recasts diffusion-prior adaptation as a principled extension of architecture-based regularization. The network is no longer merely a fixed score model; it becomes a measurement-conditioned prior that is updated across noise scales. A plausible implication is that DDIP inherits part of DIP’s scan-adaptivity while retaining the stronger generative statistics of pre-trained diffusion models.

3. Core algorithmic patterns

DDIP implementations differ in where the test-time optimization is inserted. One pattern adapts the diffusion prior itself at every reverse step. This is the strict DDIP formulation above, and it generalizes SCD by allowing any estimator of $p(x\mid y)\propto p_{\rm data}(x)p(y\mid x)$0, including DDNM-style, DPS-style, DDS-style, and DiffusionMBIR-style approximations (Chung et al., 2024). The paper emphasizes that consistency between the posterior-mean approximation used for adaptation and for sampling is important; its proof-of-concept ablation shows that simply using the same stronger estimator during adaptation improves reconstruction (Chung et al., 2024).

A second pattern uses DIP as a measurement-consistency solver inside the diffusion loop rather than directly updating the score model. In “CDDIP: Constrained Diffusion-Driven Deep Image Prior for Seismic Data Reconstruction” (Goyes-Peñafiel et al., 2024), the seismic masking model is

$p(x\mid y)\propto p_{\rm data}(x)p(y\mid x)$1

and the DIP subproblem at each diffusion step is

$p(x\mid y)\propto p_{\rm data}(x)p(y\mid x)$2

The DIP network is warm-started from the previous timestep, so the module acts as a learned-free, measurement-driven constraint rather than a standalone reconstructor (Goyes-Peñafiel et al., 2024).

A third pattern alternates DIP optimization with diffusion purification of the DIP input. In “Sequential Diffusion-Guided Deep Image Prior For Medical Image Reconstruction” (Liang et al., 2024), the DIP block solves

$p(x\mid y)\propto p_{\rm data}(x)p(y\mid x)$3

and then updates the adaptive input by

$p(x\mid y)\propto p_{\rm data}(x)p(y\mid x)$4

Here diffusion does not enforce data consistency during reverse steps; DIP handles fidelity, while the diffusion model acts as a purifier for the current reconstruction (Liang et al., 2024).

A fourth pattern alternates diffusion sampling with online fine-tuning of the score model against a physics-based likelihood. In PET reconstruction,

$p(x\mid y)\propto p_{\rm data}(x)p(y\mid x)$5

with Poisson log-likelihood

$p(x\mid y)\propto p_{\rm data}(x)p(y\mid x)$6

and the DDIP adaptation step is

$p(x\mid y)\propto p_{\rm data}(x)p(y\mid x)$7

To make this practical, the method introduces half-quadratic splitting (HQS) with alternating image and network subproblems, decoupling PET forward-model optimization from neural-network optimization (Hashimoto et al., 20 Jul 2025).

Across these variants, the invariant structure is not a particular architecture but a particular mode of inference: the prior is updated, constrained, or purified at test time in response to the specific measurement.

4. D3IP and efficient 3D adaptation

The principal systems-level development within the strict DDIP line is D3IP, introduced as an efficient adaptation method for 3D measurements (Chung et al., 2024). The motivating problem is that slice-wise DDIP or SCD adapts separate parameters $p(x\mid y)\propto p_{\rm data}(x)p(y\mid x)$8 for a volume, causing complexity to scale with the number of slices. For a $p(x\mid y)\propto p_{\rm data}(x)p(y\mid x)$9 volume, the paper reports about 6.2 hours for SCD on a single RTX 3090, whereas D3IP reduces this to about 40 minutes; LoRA parameters drop from about 2.9 GB in slice-wise SCD to about 14.5 MB with shared D3IP parameters (Chung et al., 2024).

The base D3IP objective jointly optimizes a single shared parameter set across slices: $\min_\Theta \|A\varphi_\Theta(z)-y^\delta\|^2.$0 approximated by Monte Carlo subsampling,

$\min_\Theta \|A\varphi_\Theta(z)-y^\delta\|^2.$1

Initialization uses spherical linear interpolation across the volume,

$\min_\Theta \|A\varphi_\Theta(z)-y^\delta\|^2.$2

which the paper reports as part of an initialization strategy that improves PSNR and SSIM (Chung et al., 2024).

D3IP is further integrated with 3D-aware inverse solvers. In the DiffusionMBIR-coupled version, the posterior mean approximation is written as

$\min_\Theta \|A\varphi_\Theta(z)-y^\delta\|^2.$3

where $\min_\Theta \|A\varphi_\Theta(z)-y^\delta\|^2.$4 is a finite-difference operator along the stacked slice dimension. This yields a 3D reconstruction that is explicitly regularized for inter-slice smoothness (Chung et al., 2024).

The empirical results are strongest in severe OOD settings. For 3D sparse-view CT trained on Ellipses and tested on AAPM, D3IP (mbir) reports $\min_\Theta \|A\varphi_\Theta(z)-y^\delta\|^2.$5 PSNR, $\min_\Theta \|A\varphi_\Theta(z)-y^\delta\|^2.$6 SSIM, and $\min_\Theta \|A\varphi_\Theta(z)-y^\delta\|^2.$7 LPIPS, while D3IP (meta) reports $\min_\Theta \|A\varphi_\Theta(z)-y^\delta\|^2.$8, $\min_\Theta \|A\varphi_\Theta(z)-y^\delta\|^2.$9, and $$dx_t = -t\nabla_{x_t}\log p(x_t)\,dt = \frac{x_t - [x_0|x_t]}{t}\,dt,$$0. For 3D MRI trained on Ellipses and tested on BRATS, D3IP (mbir) reaches $$dx_t = -t\nabla_{x_t}\log p(x_t)\,dt = \frac{x_t - [x_0|x_t]}{t}\,dt,$$1 PSNR and $$dx_t = -t\nabla_{x_t}\log p(x_t)\,dt = \frac{x_t - [x_0|x_t]}{t}\,dt,$$2 SSIM, substantially above DDIP and SCD; for multi-coil CS-MRI trained on fastMRI BRAIN and tested on fastMRI KNEE, D3IP (meta) reports $$dx_t = -t\nabla_{x_t}\log p(x_t)\,dt = \frac{x_t - [x_0|x_t]}{t}\,dt,$$3 PSNR, $$dx_t = -t\nabla_{x_t}\log p(x_t)\,dt = \frac{x_t - [x_0|x_t]}{t}\,dt,$$4 SSIM, and $$dx_t = -t\nabla_{x_t}\log p(x_t)\,dt = \frac{x_t - [x_0|x_t]}{t}\,dt,$$5 LPIPS (Chung et al., 2024). These results support the paper’s claim that OOD phantom-to-real adaptation is feasible without gold-standard training data.

5. Domain-specific instantiations

The broader DDIP literature has diversified rapidly across scientific and medical imaging.

Formulation	Representative setting	Reported result
CDDIP	Seismic missing-trace reconstruction	At $$dx_t = -t\nabla_{x_t}\log p(x_t)\,dt = \frac{x_t - [x_0|x_t]}{t}\,dt,$$6, about $$dx_t = -t\nabla_{x_t}\log p(x_t)\,dt = \frac{x_t - [x_0|x_t]}{t}\,dt,$$7 dB and $$dx_t = -t\nabla_{x_t}\log p(x_t)\,dt = \frac{x_t - [x_0|x_t]}{t}\,dt,$$8 SSIM on 50% irregular missing post-stack data
uDiG-DIP	MRI and CT reconstruction	MRI: $$dx_t = -t\nabla_{x_t}\log p(x_t)\,dt = \frac{x_t - [x_0|x_t]}{t}\,dt,$$9 / $[x_0|x_t] = x_t + t^2\nabla_{x_t}\log p(x_t).$0 PSNR for $[x_0|x_t] = x_t + t^2\nabla_{x_t}\log p(x_t).$1; CT: $[x_0|x_t] = x_t + t^2\nabla_{x_t}\log p(x_t).$2 / $[x_0|x_t] = x_t + t^2\nabla_{x_t}\log p(x_t).$3 PSNR for 18/30 views
PET DDIP-inspired method	Low-dose PET with anatomical prior	Clinical $[x_0|x_t] = x_t + t^2\nabla_{x_t}\log p(x_t).$4FDG mean PSNR $[x_0|x_t] = x_t + t^2\nabla_{x_t}\log p(x_t).$5, white-matter std $[x_0|x_t] = x_t + t^2\nabla_{x_t}\log p(x_t).$6, putamen CR $[x_0|x_t] = x_t + t^2\nabla_{x_t}\log p(x_t).$7
D3IP	3D OOD inverse problems	$[x_0|x_t] = x_t + t^2\nabla_{x_t}\log p(x_t).$8 volume runtime about 40 minutes vs about 6.2 hours for SCD

In seismic reconstruction, CDDIP combines a pre-trained DDPM with an Attention U-Net DIP constraint module, uses $[x_0|x_t] = x_t + t^2\nabla_{x_t}\log p(x_t).$9 diffusion timesteps with around 50 DIP steps as a good tradeoff, and is reported to outperform DiffPIR and CCSeis-DDPM on out-of-domain surveys such as Stratton, Penobscot, Blake Ridge, and Alaska. It also extends to pre-stack common shotgathers, with reported results of $[x_0|x_t]$0 dB / $[x_0|x_t]$1 on a synthetic pre-stack example and $[x_0|x_t]$2 dB / $[x_0|x_t]$3 on a field pre-stack example (Goyes-Peñafiel et al., 2024).

In MRI and CT, uDiG-DIP couples a U-Net DIP with diffusion purification. On fastMRI with Cartesian $[x_0|x_t]$4 and $[x_0|x_t]$5 undersampling, the reported PSNRs are $[x_0|x_t]$6 and $[x_0|x_t]$7, above Score-MRI, Ref-G DIP, and DIP; on AAPM CT with 18 and 30 sparse views, the reported PSNRs are $[x_0|x_t]$8 and $[x_0|x_t]$9, with SSIM $[x_0|x_t,y]$0 and $[x_0|x_t,y]$1. The method is also reported to delay overfitting to around 8000 iterations, compared with about 6000 for DIP and about 5000 for Ref-G DIP (Liang et al., 2024).

In PET, the DDIP-inspired method alternates DDIM-style sampling, online fine-tuning, and HQS-based PET image updates while conditioning on an anatomical prior $[x_0|x_t,y]$2. The method is explicitly designed for transfer across tracer and scanner shifts. In the clinical $[x_0|x_t,y]$3FDG experiment, the mean PSNR values are reported as 22.50 for MLEM, 29.33 for MAPEM, 27.44 for DPS, and 30.29 for the proposed method; mean white-matter standard deviation is 0.015 for the proposed method, below 0.121, 0.040, and 0.036 for the baselines. In the clinical $[x_0|x_t,y]$4Florbetapir case, DDIP reports 99.97% contrast and 0.220 CV, compared with 94.82%/0.736 for MLEM, 90.54%/0.300 for MAPEM, and 51.08%/0.230 for DPS (Hashimoto et al., 20 Jul 2025).

These applications collectively indicate that DDIP is not confined to a single forward model. It has been instantiated for masking/inpainting, MRI Fourier sampling, CT Radon inversion, and Poisson PET reconstruction, with adaptation carried either by score-model fine-tuning, DIP-based constraints, or sequential purification.

6. Terminological ambiguities, limitations, and related non-DDIP methods

A persistent source of confusion is that “DDIP” has also been used for “double deep image prior.” In multislice ptychography, the paper “Using a modified double deep image prior for crosstalk mitigation in multislice ptychography” uses two DIP generators for clean slices, a third DIP for mixing weights, and filtered leakage terms

$[x_0|x_t,y]$5

to model band-limited crosstalk. This is a training-free layer-separation method, not a diffusion-based prior (Du et al., 2021).

Classical DIP-based inverse methods are also conceptually related but technically distinct. “Unsupervised Image Fusion Using Deep Image Priors” reformulates image fusion as an inverse problem with

$[x_0|x_t,y]$6

and optimizes a randomly initialized CNN per image pair, but the paper explicitly states that it is not diffusion-based and has no diffusion probabilistic model, timestep schedule, noise injection/denoising chain, or diffusion training objective (Ma et al., 2021). Likewise, the physics-constrained seismic DIP that predicts a pseudo-velocity model and maps it to reflectivity through

$[x_0|x_t,y]$7

is a physics-guided analogue to DDIP rather than a diffusion model (Voytan et al., 2024).

The limitations reported across the DDIP literature are consistent. DDIP and D3IP still incur test-time adaptation overhead; D3IP is much faster than slice-wise SCD or DDIP, but not free (Chung et al., 2024). The 3D framework notes catastrophic forgetting and dependence on the quality of the chosen posterior mean approximation, as well as the need to store timestep-specific adapted parameters (Chung et al., 2024). uDiG-DIP depends on a pre-trained diffusion model, does not impose data consistency inside diffusion reverse steps, and still requires study of the number of diffusion steps $[x_0|x_t,y]$8 and DIP updates $[x_0|x_t,y]$9 (Liang et al., 2024). The PET method remains computationally heavy, currently uses a 2D network with slice stacking, and is evaluated against full-dose PET references that still contain noise (Hashimoto et al., 20 Jul 2025). CDDIP inherits DIP sensitivities such as step scheduling and per-timestep optimization cost, even though warm starting and reduced diffusion schedules improve feasibility (Goyes-Peñafiel et al., 2024).

The most stable synthesis is therefore narrow and technical: DDIP is a framework in which diffusion priors are made image-specific by test-time adaptation, constraint, or refinement, typically to address OOD inverse problems. Its significance lies in formalizing a bridge between architecture-based regularization and generative diffusion priors, while its current diversity of implementations shows that this bridge can be built through multiple algorithmic routes rather than a single canonical solver.