DDPM-PETMR: PET-MRI Fusion via Diffusion Models

• The paper introduces a diffusion-based framework that uses conditional and joint models to enhance low-dose PET reconstructions by integrating complementary MRI data.
• It employs a U-Net backbone with pixel-level concatenation of PET and MRI inputs, achieving up to +1.6 dB PSNR improvement over standard U-Net baselines.
• Extensive benchmarking on neuroimaging datasets demonstrates robust uncertainty reduction and superior structural fidelity in multi-modal PET-MRI fusion.

DDPM-PETMR refers to a class of deep generative modeling frameworks based on denoising diffusion probabilistic models (DDPM) for leveraging multi-modal positron emission tomography (PET) and magnetic resonance imaging (MRI) data. These models have emerged as prominent baselines and building blocks for PET denoising, enhancement, and fusion, facilitating high-quality PET reconstructions through the integration of complementary anatomical (MRI) and functional (PET) information. The approach encompasses both conditional DDPMs, which use co-registered PET/MRI as input, and joint generative diffusion models that model the PET–MRI distribution.

1. Theoretical Foundations: Diffusion Process and Training Objective

DDPM-PETMR employs the canonical forward–reverse Markov diffusion process introduced in [Ho et al., 2020] and widely adapted for medical imaging. The forward (noising) process perturbs a clean image $x_0$ (often a PET, MRI, or joint PET+MRI patch) by incremental addition of Gaussian noise:

$$q(x_t|x_{t-1}) = \mathcal{N}\bigl(x_t; \sqrt{1-\beta_t}\, x_{t-1}, \beta_t I\bigr)$$

$$x_t = \sqrt{\bar\alpha_t} x_0 + \sqrt{1 - \bar\alpha_t} \epsilon, \quad \bar\alpha_t = \prod_{s=1}^t (1-\beta_s),\,\, \epsilon \sim \mathcal N(0, I)$$

The reverse process is modeled by a neural network parameterization, either as a mean estimator or as a noise predictor $\epsilon_\theta$:

$$p_\theta(x_{t-1}|x_t) = \mathcal{N}\bigl(x_{t-1};\, \mu_\theta(x_t, t),\, \Sigma_\theta(x_t, t)\bigr)$$

$$\mu_\theta(x_t, t) = \frac{1}{\sqrt{\alpha_t}}\left[x_t - \frac{1-\alpha_t}{\sqrt{1-\bar\alpha_t}} \, \epsilon_\theta(x_t, t)\right]$$

Training is performed by denoising score-matching, typically using the mean-squared error in noise space:

$$\mathcal{L}_{\mathrm{DM}} = \mathbb{E}_{x_0, \epsilon, t} \Bigl[ \lVert \epsilon - \epsilon_\theta(\sqrt{\bar\alpha_t} x_0 + \sqrt{1-\bar\alpha_t} \epsilon , t) \rVert^2 \Bigr]$$

No adversarial (GAN) or perceptual losses are introduced in canonical DDPM-PETMR baselines (Yar et al., 10 Mar 2026, Gong et al., 2022).

2. Conditional and Joint Architectures for PET/MRI Input

The core DDPM-PETMR architecture is a conditional diffusion model tailored for multi-modal PET/MR inputs. The most prevalent configuration is the two-channel conditioning scheme, where low-dose PET $X$ and registered T1-weighted MRI $Z$ are concatenated to form a tensor $[X \Vert Z]$. This input, together with a time embedding $$q(x_t|x_{t-1}) = \mathcal{N}\bigl(x_t; \sqrt{1-\beta_t}\, x_{t-1}, \beta_t I\bigr)$$0, feeds into a U-Net backbone. Feature fusion is realized by simple concatenation at each encoder–decoder skip connection:

$$q(x_t|x_{t-1}) = \mathcal{N}\bigl(x_t; \sqrt{1-\beta_t}\, x_{t-1}, \beta_t I\bigr)$$1

Alternative or advanced schemes include:

• Incorporation of MR prior as a separate conditioning vector or through cross-attention (not in the original DDPM-PETMR baselines).
• Joint diffusion models, e.g., JPDDM or MC-Diffusion, model the joint distribution of PET and MRI ($$q(x_t|x_{t-1}) = \mathcal{N}\bigl(x_t; \sqrt{1-\beta_t}\, x_{t-1}, \beta_t I\bigr)$$2) and learn a joint score, enabling simultaneous sampling or mutual consistency-driven reconstruction (Xie et al., 2023, Xie et al., 2022).

Variants explored include DDPM-PET (PET only), DDPM-PETMR (PET+MR input), and MR-only networks with data-consistency enforcement during inference (Gong et al., 2022).

3. Adaptation to PET-MRI Enhancement and Reconstruction

DDPM-PETMR is applied to solve several tasks:

A. PET Denoising and Enhancement: Given a low-dose PET and auxiliary MRI, the conditional DDPM reconstructs a higher-fidelity PET estimate. Channel-wise MR input consistently improves PSNR (up to +1.6 dB over U-Net baselines) and reduces uncertainty by 20–30% (Gong et al., 2022).

B. MRI-Informed PET Synthesis:

Score-based diffusion models such as JPDDM and MC-Diffusion learn the joint/probabilistic distribution over PET–MRI and can generate PET from MRI alone or co-reconstruct both via coupled predictor–corrector SDE samplers. These models allow flexible enforcement of data-fidelity (e.g., PET sinogram consistency) and mutual consistency (MRI k-space alignment) (Xie et al., 2023, Xie et al., 2022).

C. Pseudo-PET/MRI Guidance:

Advanced variants generate subject-specific pseudo-PET priors via registration-based image warping and ensemble averaging, then condition diffusion priors on this synthesized anatomy for improved generalization and preservation of PET-unique features (Webber et al., 4 Jun 2025).

D. Multi-Modality Feature Fusion and OOD Robustness:

Recent works implement transformer or invertible coupling-based fusion modules and two-stage (in-distribution/out-of-distribution) training to further harness multi-modal structural and intensity information (Zhang et al., 12 Feb 2026).

4. Experimental Protocols and Benchmarking

DDPM-PETMR and its derivatives have been extensively validated on human neuroimaging datasets, including DaCRA (healthy subjects), ADNI (Alzheimer’s Disease), and multimodal tracer studies (Yar et al., 10 Mar 2026, Gong et al., 2022, Webber et al., 4 Jun 2025). The standard experimental protocol involves:

• 2D axial slice-wise training with normalization to [0,1] or SUV scaling; patch size typically 256×256 or 192×192.
• Diffusion steps $$q(x_t|x_{t-1}) = \mathcal{N}\bigl(x_t; \sqrt{1-\beta_t}\, x_{t-1}, \beta_t I\bigr)$$3; Adam(W) optimizer, learning rate $$q(x_t|x_{t-1}) = \mathcal{N}\bigl(x_t; \sqrt{1-\beta_t}\, x_{t-1}, \beta_t I\bigr)$$4, batch size 1–4.
• Quantitative metrics: SSIM, PSNR, NMSE, and LPIPS (for perceptual similarity).
• Baselines: Nonlocal means (NLM), GANs (CycleWGAN, Pix2PixHD), U-Net, and contrastive or multi-branch diffusion models.
• State-of-the-art variants, e.g., M2Diff, MFdiff, and MC-Diffusion, outperform DDPM-PETMR specifically in challenging multi-task, OOD, or joint fusion settings.

Sample comparative metrics (DaCRA ×100) (Yar et al., 10 Mar 2026):

Method	SSIM	PSNR	NMSE	LPIPS
DDPM-PETMR	0.9462	28.10	0.0646	0.0378
M2Diff	0.9528	28.64	0.0694	0.0349
Multi-branch UNet	0.9498	28.50	0.0648	0.0370

Consistent trends show that DDPM-PETMR forms a robust baseline, but advanced models yield sharper reconstructions and better structure preservation.

5. Uncertainty Quantification, Bias, and Limitations

Uncertainty and model bias are critical for clinical PET/MR applications:

• Voxelwise uncertainty maps can be produced via test-time sampling, indicating model confidence.
• Conditioning on MRI in DDPM-PETMR consistently lowers local and global uncertainty, though over-reliance may introduce bias, particularly in high-uptake regions where PET and MR contrast diverge.
• MR-only denoising models (without PET input) yield visually crisp but sometimes anatomically biased images, hallucinating MR-guided detail where PET physiology is decoupled (e.g., deep nuclei) (Gong et al., 2022).

Key limitations include early feature dilution from shallow PET+MR fusion, lack of explicit multi-task constraints, fixed variance schedules (constraining noise modeling flexibility), and limited evaluation on diverse or multi-scanner datasets.

6. Extensions, Comparative Models, and Current Research Trends

Subsequent developments center on overcoming the above limitations:

• Hierarchical feature fusion (HFF) and multi-task separation (e.g., parallel PET and MRI streams with late fusion) (Yar et al., 10 Mar 2026).
• Explicit learning of variance in the reverse process (IDDPM, DiffusionMTL).
• Transformer-based feature fusion and invertible coupling layers to optimize MR–PET fusion and OOD robustness (Zhang et al., 12 Feb 2026).
• Training and fine-tuning on simulated phantoms and in vivo OOD scenarios to learn both generalized and specific priors.
• Joint diffusion score models for simultaneous PET and MR reconstruction, advancing both noise suppression and artifact reduction beyond conventional architectures (Xie et al., 2023).
• Anatomical prior tuning post hoc, enabling PET enhancement even in the absence of paired, high-quality MRI (Gan et al., 2024, Webber et al., 4 Jun 2025).

A plausible implication is that future DDPM-PETMR frameworks will increasingly move toward highly modular, task-specific, and uncertainty-aware architectures, combining subject-specific anatomical priors, joint diffusion samplers, and multi-scenario pretraining/fine-tuning.

7. Significance and Outlook

DDPM-PETMR and its conceptual successors represent a transition from deterministic, single-modality denoising to high-fidelity, probabilistic modeling of the coupled PET–MR data space. By fully leveraging diffusion-based generative modeling and multi-modal fusion, these methods improve PET noise suppression, anatomical fidelity, and the handling of uncertainty and OOD data, leading to more reliable quantitative imaging and downstream diagnostics (Yar et al., 10 Mar 2026, Webber et al., 4 Jun 2025, Zhang et al., 12 Feb 2026).

These diffusion-based frameworks now constitute the reference baseline against which emerging PET/MR fusion and enhancement approaches are quantitatively benchmarked, defining the current state of the art in multi-modal PET reconstruction.