Multi-Dynamic Low-Rank Deep Image Prior (ML-DIP)

• The paper introduces ML-DIP, a method that integrates multi-dynamic modeling, low-rank representation, and deep image priors to reconstruct dynamic 3D CMR images without external training data.
• The architecture separates static CNNs for image and deformation bases from lightweight frame-specific networks, efficiently capturing both motion and non-motion dynamics.
• Empirical results demonstrate high PSNR/SSIM benchmarks and improved performance over fixed-template and 5D-Cine methods in challenging real-time cardiac MRI scenarios.

Searching arXiv for recent and foundational papers on ML-DIP and closely related low-rank/DIP formulations. Multi-Dynamic Low-Rank Deep Image Prior (ML-DIP) is a scan-specific, training-free reconstruction framework for highly undersampled dynamic imaging that combines explicit low-rank modeling with deep image prior (DIP) parameterization. In the formulation explicitly named “ML-DIP,” developed for 3D real-time cine cardiovascular magnetic resonance (CMR), the method reconstructs a full 3D+time image series from free-breathing, ungated k-space data by modeling spatial image content and temporal deformation fields with separate neural networks optimized per scan, without external fully sampled training data (Chen et al., 25 Jul 2025). More broadly, related literature suggests a class of hybrid methods in which low-rank structure captures global temporal, spectral, or multi-way correlations while DIP-type or plug-and-play deep priors encode local or scan-adaptive image statistics; this broader interpretation is consistent with earlier tensor-completion, dynamic MRI, and embedded-manifold formulations (Zhao et al., 2019).

1. Definition and conceptual scope

The defining feature of ML-DIP is the joint use of three ingredients: multi-dynamic modeling, low-rank representation, and deep image priors. In the 3D real-time cine CMR formulation, “multi-dynamic” denotes simultaneous handling of multiple sources of temporal variation, including cardiac motion, respiratory motion, non-rigid and bulk torso motion, and non-motion dynamics such as contrast changes. “Low-rank” denotes a factorized representation of the dynamic series through a static deformation basis, a static image basis, and frame-specific low-dimensional weights. “Deep image prior” denotes the use of randomly initialized neural networks optimized only on the undersampled data of a single scan, with no external training dataset and no pre-trained model (Chen et al., 25 Jul 2025).

This combination distinguishes ML-DIP from several neighboring paradigms. Standard DIP or dynamic DIP generates frames directly from an untrained convolutional network and is described as under-constrained for 3D real-time imaging with $R>1000$. Motion-only formulations such as MoCo-SToRM and DMoCo model each frame as a deformation of a single 3D template, whereas ML-DIP adds a CNN-based image basis to represent contrast and content dynamics in addition to deformation. LR-DIP is described as using low-rank modeling of images with a DIP but only implicit motion modeling; ML-DIP instead models motion explicitly through deformation fields (Chen et al., 25 Jul 2025).

A broader reading of the term is suggested by earlier work that combines low-rank tensor modeling with deep priors for multi-dimensional reconstruction. “Deep Plug-and-play Prior for Low-rank Tensor Completion” formulates a hybrid model in which tensor nuclear norm (TNN) captures global low-rank structure and a deep denoiser encodes an implicit deep image prior on spatial slices, thereby establishing a concrete low-rank-plus-deep-prior template for multi-dimensional image completion (Zhao et al., 2019). In dynamic MRI, “Dynamic MRI Reconstruction Via Dual Deep Priors and Low-Rank Plus Sparse Modeling” realizes a closely related idea by parameterizing low-rank background and sparse dynamics with two untrained DIPs and optimizing them jointly with explicit nuclear-norm and $\ell_1$ penalties (Sun et al., 18 May 2026).

2. Mathematical formulation and low-rank representation

In the 3D cine CMR setting, the dynamic image series is denoted

$x^{(1:T)} \coloneq \{x^{(t)}\}_{t=1}^T,$

with each frame $x^{(t)}\in \mathbb{C}^{N\times 1}$. The measurement model is

$y^{(t)} = A^{(t)} x^{(t)} + \epsilon^{(t)},$

where $y^{(t)}$ is measured multi-coil k-space data, $\epsilon^{(t)}$ is additive white Gaussian noise, and $A^{(t)} = SFC$ combines coil sensitivities, 3D Fourier transform, and time-varying sampling (Chen et al., 25 Jul 2025).

Instead of directly solving a regularized least-squares problem over all frames, ML-DIP represents each frame through a frame-specific composite image $c^{(t)}$ and a frame-specific deformation field $\phi^{(t)}$, generating the predicted frame by

$\ell_1$0

where $\ell_1$1 denotes a 3D spatial warping operation implemented via a differentiable grid sampler or spatial transformer (Chen et al., 25 Jul 2025).

The low-rank factorization is introduced through two static bases. The deformation basis is

$\ell_1$2

and the image basis is

$\ell_1$3

For each frame $\ell_1$4, ML-DIP learns deformation weights $\ell_1$5 and image weights $\ell_1$6, so that

$\ell_1$7

and therefore

$\ell_1$8

The low-rank character lies in the fact that all temporal variability is mediated by time-varying coefficients on small static bases rather than by independent per-frame volumes or deformations (Chen et al., 25 Jul 2025).

This factorization has direct analogues in related low-rank deep-prior formulations. In the dual-DIP dynamic MRI model, the entire cine sequence is stacked into a Casorati matrix $\ell_1$9 and decomposed as

$x^{(1:T)} \coloneq \{x^{(t)}\}_{t=1}^T,$0

with $x^{(1:T)} \coloneq \{x^{(t)}\}_{t=1}^T,$1 low-rank and $x^{(1:T)} \coloneq \{x^{(t)}\}_{t=1}^T,$2 sparse; both components are parameterized by separate untrained 3D CNNs and jointly regularized by nuclear norm and transform sparsity (Sun et al., 18 May 2026). In tensor completion, the corresponding global low-rank prior is expressed via TNN under the t-SVD framework,

$x^{(1:T)} \coloneq \{x^{(t)}\}_{t=1}^T,$3

which promotes low multi-rank across Fourier slices while a deep denoiser regularizes local image structure (Zhao et al., 2019).

3. Network architecture, latent parameterization, and optimization

The ML-DIP architecture consists of four subnetworks and three sets of latent codes. A ConvDecoder $x^{(1:T)} \coloneq \{x^{(t)}\}_{t=1}^T,$4 maps a static 3D code $x^{(1:T)} \coloneq \{x^{(t)}\}_{t=1}^T,$5 to the deformation basis $x^{(1:T)} \coloneq \{x^{(t)}\}_{t=1}^T,$6. A 3D U-Net $x^{(1:T)} \coloneq \{x^{(t)}\}_{t=1}^T,$7 maps a static 3D code $x^{(1:T)} \coloneq \{x^{(t)}\}_{t=1}^T,$8 to the image basis $x^{(1:T)} \coloneq \{x^{(t)}\}_{t=1}^T,$9. Two small fully connected networks, $x^{(t)}\in \mathbb{C}^{N\times 1}$0 and $x^{(t)}\in \mathbb{C}^{N\times 1}$1, take a frame-specific latent vector $x^{(t)}\in \mathbb{C}^{N\times 1}$2 and produce $x^{(t)}\in \mathbb{C}^{N\times 1}$3 and $x^{(t)}\in \mathbb{C}^{N\times 1}$4, respectively (Chen et al., 25 Jul 2025).

This decomposition separates large time-invariant convolutional generators from lightweight temporal parameterization. The large CNNs are static across time and therefore do not scale with $x^{(t)}\in \mathbb{C}^{N\times 1}$5 or batch size, while temporal variation is captured by small FC networks and the sequence of latent codes $x^{(t)}\in \mathbb{C}^{N\times 1}$6. The design is explicitly motivated as a memory- and computation-efficient way to model thousands of frames under extreme undersampling (Chen et al., 25 Jul 2025).

The optimization problem is

$x^{(t)}\in \mathbb{C}^{N\times 1}$7

where the only explicit regularizer is a deformation smoothness term given by the sum over time of squared spatial finite differences of $x^{(t)}\in \mathbb{C}^{N\times 1}$8 along all three spatial directions. No explicit regularization is imposed on $x^{(t)}\in \mathbb{C}^{N\times 1}$9; the regularity of composite images is attributed to the U-Net architecture and the low-rank representation (Chen et al., 25 Jul 2025).

Training is performed with Adam for 48,000 joint iterations using a cosine annealing learning-rate schedule from $y^{(t)} = A^{(t)} x^{(t)} + \epsilon^{(t)},$0 to $y^{(t)} = A^{(t)} x^{(t)} + \epsilon^{(t)},$1. A batch size of 20 contiguous frames is used as a memory-saving device. The total parameter count is approximately 17 million (Chen et al., 25 Jul 2025). Static codes $y^{(t)} = A^{(t)} x^{(t)} + \epsilon^{(t)},$2 and $y^{(t)} = A^{(t)} x^{(t)} + \epsilon^{(t)},$3 are initialized iid from a uniform distribution. The frame-specific codes $y^{(t)} = A^{(t)} x^{(t)} + \epsilon^{(t)},$4 are initialized by self-gating: a repeatedly acquired central k-space line is bandpass filtered, principal component analysis is applied, and six dominant components—two respiratory and four cardiac—form the initialization for the latent trajectory (Chen et al., 25 Jul 2025).

A different but instructive low-dimensional strategy appears in “Image Reconstruction via Deep Image Prior Subspaces,” which constrains DIP optimization to an affine low-dimensional linear subspace of parameter space,

$y^{(t)} = A^{(t)} x^{(t)} + \epsilon^{(t)},$5

where $y^{(t)} = A^{(t)} x^{(t)} + \epsilon^{(t)},$6 is obtained from a truncated SVD of pretraining trajectories and $y^{(t)} = A^{(t)} x^{(t)} + \epsilon^{(t)},$7 is a leverage-score sparsification mask. That construction is presented as a way to reduce DIP overfitting and enable stable second-order optimization, and it provides a complementary notion of low-rank structure in DIP—low-dimensionality in parameter space rather than low-rank factorization of the reconstructed dynamics (Barbano et al., 2023).

4. Relation to deep image prior, plug-and-play, and embedded-space interpretations

ML-DIP belongs to the family of physics-driven DIP methods. The MRI forward model is embedded directly in the loss, and the networks are optimized per scan from random initialization rather than learned offline from large datasets. In that sense, the framework follows the strict DIP philosophy: the prior is scan-specific and arises from the optimization dynamics of structured networks fitted to a single acquisition (Chen et al., 25 Jul 2025).

The literature presents several distinct but related realizations of “deep prior plus low-rank structure.” In DP3LRTC, a pre-trained CNN denoiser is used within an ADMM-based plug-and-play framework, while TNN enforces global tensor low-rankness. The combined model

$y^{(t)} = A^{(t)} x^{(t)} + \epsilon^{(t)},$8

explicitly separates global inter-slice structure from local spatial detail. The paper notes that its CNN is pre-trained and fixed during reconstruction, so it is not a “true” DIP in the original per-instance sense, but it is still presented as embodying a deep image prior learned from natural images (Zhao et al., 2019).

The dual-DIP dynamic MRI model is closer to classical DIP. It uses two untrained 3D CNN generators with fixed random Gaussian inputs,

$y^{(t)} = A^{(t)} x^{(t)} + \epsilon^{(t)},$9

and solves

$y^{(t)}$0

That formulation is described as a structured DIP framework for dynamic MRI reconstruction that explicitly models spatiotemporal correlations through low-rank plus sparse decomposition and establishes a convergence analysis for extrapolated ADMM in the presence of DIP-based nonconvex parameterizations (Sun et al., 18 May 2026).

A third interpretive lineage comes from “Manifold Modeling in Embedded Space: A Perspective for Interpreting Deep Image Prior,” which argues that DIP can be understood as a low-dimensional patch-manifold prior. MMES makes this explicit through multi-way delay embedding $y^{(t)}$1, an autoencoder $y^{(t)}$2, and the reconstruction model

$y^{(t)}$3

This suggests an interpretation of ML-DIP-type methods in which low-rankness, low-dimensional manifolds, and untrained network inductive bias are not competing explanations but alternative parameterizations of structured low-complexity image dynamics (Yokota et al., 2019).

5. Empirical performance in 3D real-time cardiovascular MRI

The principal empirical validation of ML-DIP is in 3D real-time cine CMR reconstructed from ordered pseudo-radial sampling (OPRA) with acceleration factors exceeding 1,000. The acquisition uses full sampling along $y^{(t)}$4 and 11 readouts per 3D frame in the $y^{(t)}$5–$y^{(t)}$6 plane, with the 6th readout fixed at $y^{(t)}$7 for self-gating. Healthy-subject and patient studies use matrix size $y^{(t)}$8, temporal resolution 32–34 ms, and $y^{(t)}$9 with approximately 9000 frames over 5 minutes (Chen et al., 25 Jul 2025).

In the MRXCAT phantom with simulated premature ventricular contractions (PVCs), ML-DIP was evaluated over varying scan durations. The reported PSNR/SSIM values were $\epsilon^{(t)}$0 at 300 s, $\epsilon^{(t)}$1 at 240 s, $\epsilon^{(t)}$2 at 180 s, $\epsilon^{(t)}$3 at 120 s, $\epsilon^{(t)}$4 at 60 s, $\epsilon^{(t)}$5 at 30 s, $\epsilon^{(t)}$6 at 12 s, and $\epsilon^{(t)}$7 at 3.4 s. The study reports that image quality remains high down to about 2 minutes and still good at 1 minute, whereas quality drops more sharply below about 30 seconds. Visual evaluation indicates recovery of respiratory motion, cardiac motion, and clear PVC beats in $\epsilon^{(t)}$8-$\epsilon^{(t)}$9 profiles. A fixed-template ablation, in which all frames are constrained to be deformations of a single image, is reported to show substantial distortions and failure to model contrast or dynamic content (Chen et al., 25 Jul 2025).

In healthy volunteers, including exercise acquisitions, ML-DIP is reported to yield left-ventricular functional measurements comparable to 2D real-time cine and higher image quality than 5D-Cine. Expert image-quality scores averaged 4.88 for ML-DIP, 4.48 for 5D-Cine, and 4.23 for 2D real-time at rest; under exercise the averages were 4.63 for ML-DIP, 3.13 for 5D-Cine, and 4.25 for 2D real-time. Across 17 acquisitions, mean absolute differences between ML-DIP and 2D real-time were 7.41 mL for EDV, 5.43 mL for ESV, 5.75 mL for SV, and 2.65% for EF, with Pearson correlations greater than 0.9 for all metrics (Chen et al., 25 Jul 2025).

In five patients with frequent PVCs, ML-DIP is reported to preserve beat-to-beat variability and reconstruct irregular beats and compensatory pauses, whereas 5D-Cine often exhibits severe artifacts and information loss due to binning. Image-quality scores in this cohort averaged 4.10 for ML-DIP, 2.40 for 5D-Cine, and 4.50 for 2D real-time. ML-DIP enabled left-ventricular function quantification in all five PVC patients, while 5D-Cine was adequate for quantification in only two of five (Chen et al., 25 Jul 2025).

The broader literature shows analogous empirical gains when low-rank structure is paired with deep priors in other modalities. DP3LRTC reports average PSNR gains over TNN-3DTV greater than 3.9 dB and SSIM gains greater than 0.05 on color-image completion, and it reports superior performance on grayscale video and multispectral image completion as well. The dual-DIP dynamic MRI model reports that its proposed method consistently outperforms classical reconstruction and existing supervised and unsupervised MRI reconstruction techniques, with an ablation in which a single-network DIP gives about 31.23 dB, low-rank-only DIP about 29.82 dB, sparse-only DIP about 30.18 dB, and the proposed dual DIP with L+S about 33.83 dB (Zhao et al., 2019).

6. Limitations, interpretations, and future directions

Several limitations recur across ML-DIP and closely related formulations. For the 3D CMR method, the authors explicitly note small sample size, long reconstruction time of about 7–10 hours per 5-minute dataset on a single Nvidia RTX 6000 Ada GPU, incomplete validation of advanced functional metrics, and the absence of exhaustive hyperparameter or architecture optimization. The reconstruction is therefore not real-time at present, although inference after training is fast: about 2–3 s for 100 phantom frames and 5–8 s for 200 in vivo frames (Chen et al., 25 Jul 2025).

A common misconception is that low-rank deep-prior methods necessarily reduce dynamics to motion of a single template. The ML-DIP ablation against a fixed-template version directly contradicts that simplification: the method’s image basis $A^{(t)} = SFC$0 and frame-specific coefficients $A^{(t)} = SFC$1 are introduced precisely to model non-motion dynamics such as contrast changes, inflow effects, saturation, and other intensity variations that cannot be represented by deformation alone (Chen et al., 25 Jul 2025). Related dynamic MRI work reaches an analogous conclusion by separating low-rank background and sparse innovations into two DIPs rather than forcing a monolithic sequence generator (Sun et al., 18 May 2026).

Another recurring issue is the status of convergence and overfitting. In plug-and-play tensor completion, formal convergence theory with general deep denoisers is described as incomplete, though numerically stable convergence curves are observed. In contrast, the dual-DIP L+S dynamic MRI model provides a Lyapunov-based convergence analysis showing sufficient descent and critical-point convergence of cluster points under stated assumptions. Subspace-DIP work approaches the same problem from a different angle by restricting optimization to a low-dimensional affine parameter subspace, which reduces overfitting and makes loss-based stopping more reliable (Zhao et al., 2019).

The literature also indicates multiple plausible extensions. The 3D CMR ML-DIP paper proposes pre-training followed by scan-specific fine-tuning, hierarchical or multi-resolution training, broader validation across pathologies, and enhanced motion models with additional constraints on deformations or latent trajectories. Earlier low-rank-plus-deep-prior papers suggest other directions: adaptive or non-convex tensor-rank surrogates, higher-order tensor priors such as WSTNN, 3D or 4D CNNs instead of slice-wise priors, hybrid plug-and-play–DIP formulations, and explicit manifold regularization in delay-embedded spaces (Chen et al., 25 Jul 2025).

Taken together, these works indicate that ML-DIP is both a specific 3D real-time CMR reconstruction method and a broader methodological pattern: scan-specific deep priors can be made substantially more effective when coupled to explicit low-rank structure that organizes dynamics across time, spectrum, or tensor modes. This suggests a general design principle rather than a single architecture—global dynamics are constrained by low-rank factorization, while local or scan-adaptive detail is supplied by untrained or implicit neural priors (Yokota et al., 2019).