Deep Dynamic Image Prior (D2IP)

• The paper introduces D2IP as a framework that optimizes untrained neural networks per instance to enforce temporal and spatiotemporal consistency in dynamic image reconstructions.
• It incorporates techniques like unsupervised parameter warm-start, latent trajectory mapping, and dual-component (L+S) decomposition to enhance reconstruction fidelity and efficiency.
• Experimental results demonstrate significant improvements in metrics (MSSIM, PSNR, RSNR) and reduced reconstruction times for applications such as dynamic MRI and 3D pulmonary EIT.

The Deep Dynamic Image Prior (D2IP) framework refers to a class of unsupervised, training-data-free reconstruction algorithms for dynamic imaging inverse problems, distinguished by their use of implicit neural representation priors optimized per instance and mechanisms that enforce temporal or spatiotemporal coherence across a sequence of images. D2IP emerged as a response to the limitations of classical Deep Image Prior (DIP) approaches in dynamic or high-dimensional settings, combining the adaptivity and generalization of DIP with architectural, optimization, and/or explicit structural modifications tailored for time-sequence or time-varying imaging scenarios. Initial applications were developed independently for dynamic MRI reconstruction and 3D time-sequence electrical impedance tomography (EIT), establishing its utility across diverse modalities (Yoo et al., 2019, Fang et al., 18 Jul 2025, Sun et al., 18 May 2026).

1. Fundamental Principles and Motivation

D2IP frameworks originated to address two major challenges in dynamic imaging: the scarcity of labeled dynamic datasets and the need for temporal consistency in reconstructions under extreme data undersampling or ill-posedness. Standard supervised deep learning methods are often inapplicable due to the lack of comprehensive temporal ground truth, and conventional unsupervised approaches, while data-agnostic, tend to reconstruct temporally inconsistent frames or yield poor spatiotemporal fidelity.

Core principles of D2IP include:

• Implicit prior via untrained deep networks: Instead of learning a mapping from data, reconstructions are generated by optimizing a randomly initialized neural network’s parameters (or latent input) to fit measured data for each instance, enforcing powerful structural priors.
• Temporal regularization or coupling: Frame-by-frame reconstructions are coupled either via shared or smoothly-evolving representations (e.g., latent trajectory, parameter propagation, or explicit low-rank/sparse priors).
• Optimization-based instance fitting: All model parameters are directly fitted to each measurement sequence, obviating the need for training datasets and yielding scan-specific solutions.

These properties render D2IP highly applicable in clinical and scientific contexts where sample-specific adaptation and data efficiency are essential.

2. Mathematical Formulations and Temporal Coupling Mechanisms

Several mathematical formulations of D2IP have emerged, depending on application and prior structure.

Time-Indexed Parameterization (EIT, pulmonary imaging) (Fang et al., 18 Jul 2025)

Each time frame $i$ is reconstructed as:

$$\hat{\Delta \boldsymbol{\sigma}_i} = \mathcal{V}\left(\phi(\boldsymbol{\theta}_i \mid \mathbf{Z})\right),$$

where $\phi$ (3D-FastResUNet) is optimized per frame, with loss:

$$\mathcal{L}_i = \|\Delta\mathbf{v}_i - \mathbf{J}\mathcal{V}(\phi(\boldsymbol{\theta}_i|\mathbf{Z}))\| + \lambda_{\mathrm{TV}}\mathcal{R}_{\mathrm{4D-TV}}.$$

Temporal coupling uses unsupervised parameter warm-start (UPWS) and temporal parameter propagation (TPP):

• UPWS: $$\boldsymbol{\theta}_1^{(0)} = \boldsymbol{\theta}_0^{(N_0)}$$
• TPP: $$\boldsymbol{\theta}_{i+1}^{(0)} = \boldsymbol{\theta}_i^{(N_i)}$$ So parameter optimization for frame $i+1$ begins from frame $i$’s optimal weights, enforcing temporal smoothness.

Trajectory in Latent Space (dynamic MRI) (Yoo et al., 2019)

A single generator $f_\theta$ produces all frames from a latent trajectory $\{\mathbf{z}_k\}$:

$$\hat{\Delta \boldsymbol{\sigma}_i} = \mathcal{V}\left(\phi(\boldsymbol{\theta}_i \mid \mathbf{Z})\right),$$0

$$\hat{\Delta \boldsymbol{\sigma}_i} = \mathcal{V}\left(\phi(\boldsymbol{\theta}_i \mid \mathbf{Z})\right),$$1 are arranged on a fixed low-dimensional manifold (line, circle, helix) encoding temporal proximity, e.g.:

$$\hat{\Delta \boldsymbol{\sigma}_i} = \mathcal{V}\left(\phi(\boldsymbol{\theta}_i \mid \mathbf{Z})\right),$$2

This construction imposes temporal smoothness, periodicity, and continuity in the reconstructed sequence.

Dual-Component Decomposition (dynamic MRI, L+S) (Sun et al., 18 May 2026)

Dynamic sequence $$\hat{\Delta \boldsymbol{\sigma}_i} = \mathcal{V}\left(\phi(\boldsymbol{\theta}_i \mid \mathbf{Z})\right),$$3 is decomposed as $$\hat{\Delta \boldsymbol{\sigma}_i} = \mathcal{V}\left(\phi(\boldsymbol{\theta}_i \mid \mathbf{Z})\right),$$4 (low-rank+ sparse), with each component parameterized by an independent DIP:

$$\hat{\Delta \boldsymbol{\sigma}_i} = \mathcal{V}\left(\phi(\boldsymbol{\theta}_i \mid \mathbf{Z})\right),$$5

The low-rank prior captures shared background; the sparse prior models dynamics.

3. Network Architectures and Efficiency Strategies

D2IP employs particular architectural innovations to ensure efficiency and scalability for high-dimensional dynamic settings:

• 3D-FastResUNet (Fang et al., 18 Jul 2025): An encoder–decoder with stem block, six 3D ResConv blocks, SE units, ASPP, attention, and tail block with depthwise separable 3D convolutions (DWSepConv3D), achieving significant reduction in computational cost. Depthwise separable convolutions factorize a $$\hat{\Delta \boldsymbol{\sigma}_i} = \mathcal{V}\left(\phi(\boldsymbol{\theta}_i \mid \mathbf{Z})\right),$$6 operation into $$\hat{\Delta \boldsymbol{\sigma}_i} = \mathcal{V}\left(\phi(\boldsymbol{\theta}_i \mid \mathbf{Z})\right),$$7, reducing parameter and memory footprint.
• Mapping network + generator (Yoo et al., 2019): Two FC layers ($$\hat{\Delta \boldsymbol{\sigma}_i} = \mathcal{V}\left(\phi(\boldsymbol{\theta}_i \mid \mathbf{Z})\right),$$8 hidden, ReLU) followed by an untrained convolutional generator that upsamples an $$\hat{\Delta \boldsymbol{\sigma}_i} = \mathcal{V}\left(\phi(\boldsymbol{\theta}_i \mid \mathbf{Z})\right),$$9 latent code through 14 Conv+BN+ReLU blocks with nearest-neighbor upsampling, producing complex-valued outputs.
• 3D U-Net DIPs for L+S decomposition (Sun et al., 18 May 2026): Two separate 3D U-Nets synthesize low-rank and dynamic components, each from a fixed spatial-temporal noise input, optimized per scan.

Strong regularization is provided by architectural structure, choice of priors, and measurement-domain losses, reducing overfitting despite the absence of explicit data-driven training.

4. Experimental Results and Comparative Evaluation

D2IP frameworks demonstrate state-of-the-art performance across modalities and metrics:

Method	Avg MSSIM / PSNR	ERR	Speedup	Reference
D2IP (tsEIT, pulm.)	+24.8% MSSIM	-8.1%	7.1×	(Fang et al., 18 Jul 2025)
D2IP (dynMRI, single-nt)	28.05 dB RSNR	–	–	(Yoo et al., 2019)
D2IP (dynMRI, dual L+S)	29.74 dB / .90	–	–	(Sun et al., 18 May 2026)

In 3D time-sequence EIT lung imaging (Fang et al., 18 Jul 2025), D2IP achieves a mean MSSIM of 0.7332, ERR 0.7067, and mean PSNR 10.92—superior to Tikhonov, TV, and Shallow Image Prior baselines. 20-frame reconstruction time is reduced from 595–960 s (Tikhonov/TV/R-SIP) to 135 s.

For dynamic cardiac MRI (Yoo et al., 2019), D2IP with "helix + MapNet" manifold attains 28.05 dB RSNR, outperforming GRASP, BP, and other unsupervised and model-based approaches. L+S D2IP (Sun et al., 18 May 2026) further improves PSNR/SSIM and approaches supervised L+S-Net performance, with 3 dB gain over INR/DIP methods at high acceleration.

Ablation studies consistently show that temporal coupling (e.g., TPP, latent manifold, dual DIP) is critical for rapid convergence and spatiotemporal fidelity.

5. Datasets, Benchmarks, and Application Scope

D2IP has been evaluated on challenging simulated and clinical datasets:

• tsEIT: COMSOL-based thoracic phantoms (healthy expansion and pulmonary edema), 3D time-sequence, 32 dual-layer electrodes, 328 boundary measurements per frame, and clinical measurements in two human subjects with CT-derived thoracic anatomy (Fang et al., 18 Jul 2025).
• Cardiac MRI: OCMR public dataset, variable-density undersampling, 26 cine slices, 20–30 frames/slice, multi-coil encoding, acceleration factors up to 8× (Sun et al., 18 May 2026); real and retrospective fetal cardiac MRI without heartbeat labeling (Yoo et al., 2019).

Baselines include Tikhonov and TV regularization, shallow image prior models, L+S-Net, compressed sensing, Fourier-feature implicit neural representations, and classical and recent DIP variants.

6. Clinical and Scientific Implications

D2IP enables practical application of advanced 3D dynamic imaging in clinical contexts previously considered infeasible due to computational or data constraints. The 3D-optimized, unsupervised and temporally consistent reconstructions are beneficial for:

• Pulmonary imaging: Fast, bedside-capable, radiation-free monitoring of dynamic lung ventilation, enabling assessment of asymmetric airflow, edema, and tracking of physiological or pathological changes without the need for extensive labeled 3D datasets (Fang et al., 18 Jul 2025).
• Dynamic MRI: High-fidelity cardiac and fetal motion imaging under heavy undersampling, without heartbeat annotation, suited for human and animal studies with limited access to ground truth or large training datasets (Yoo et al., 2019, Sun et al., 18 May 2026).

A plausible implication is broad applicability to other time-varying inverse problems, such as functional imaging, dynamic microscopy, or geophysical monitoring, where training data are sparse but temporal regularization is essential.

7. Historical Development and Related Approaches

The D2IP family extends DIP (Deep Image Prior), first targeting static images, to sequences by introducing temporal coupling. Early efforts relied on shared weights or unstructured network parameter sharing (Yoo et al., 2019). Later, methods such as dual DIPs with L+S modeling, eADMM optimization (Sun et al., 18 May 2026), and parameter-propagation/warm-start (Fang et al., 18 Jul 2025) addressed scalability and dynamic consistency in 3D and high-dimensional settings.

D2IP is related to manifold-constrained priors, implicit neural representations, scan-adaptive regularization, and hybrid model-based/data-driven inverse problem strategies. It remains distinct from direct supervised learning, as no population training is invoked.

---

References:

• (Fang et al., 18 Jul 2025) D2IP: Deep Dynamic Image Prior for 3D Time-sequence Pulmonary Impedance Imaging
• (Sun et al., 18 May 2026) Dynamic MRI Reconstruction Via Dual Deep Priors and Low-Rank Plus Sparse Modeling
• (Yoo et al., 2019) Time-Dependent Deep Image Prior for Dynamic MRI