SiM2P: Simulated MRI-to-PET Imaging

• SiM2P is a 3D generative framework that synthesizes clinical-grade FDG-PET images from structural MRI and auxiliary patient data to support improved dementia diagnosis.
• It employs a probabilistic diffusion bridge model with a 3D diffusion Transformer, leveraging paired MRI–PET data and multiple covariates for accurate functional mapping.
• Clinical evaluations show SiM2P improves diagnostic accuracy, confidence, and interrater agreement over MRI alone, demonstrating its practical utility in distinguishing neurodegenerative disorders.

SiM2P, short for Simulated MRI-to-PET, is a 3D generative framework that synthesizes clinical-grade FDG-PET from routinely available structural MRI plus auxiliary patient information, with the explicit aim of improving dementia diagnosis when real PET is unavailable, expensive, or impractical. It is formulated as a probabilistic structural-to-functional mapping rather than a deterministic image translation model: the framework learns a conditional diffusion bridge from MRI and covariates to FDG-PET, and is positioned for diagnostic support in differentiating Alzheimer’s disease, behavioral-variant frontotemporal dementia, and cognitively healthy controls (Li et al., 17 Oct 2025).

1. Clinical scope and problem setting

SiM2P is motivated by a specific diagnostic asymmetry. FDG-PET is described as more sensitive than MRI to early neurodegenerative hypometabolism, particularly for distinguishing Alzheimer’s disease (AD), behavioral-variant frontotemporal dementia (bvFTD), and cognitively healthy controls, yet PET remains less accessible because of cost, scanner availability, and radiation exposure (Li et al., 17 Oct 2025). SiM2P addresses this by attempting to make PET-like metabolic information available from standard MRI workflows.

The framework is trained on paired MRI–PET data. Its principal imaging input is a 3D T1-weighted MRI volume, and it is additionally conditioned on auxiliary patient information. During training, the target modality is FDG-PET. The auxiliary conditioning can include up to 13 variables: age, gender, education, MMSE, ADAS-Cog-13, ApoE4, and MRI-derived FreeSurfer segmentation measures such as CSF, gray matter, white matter, bilateral hippocampal volumes, and bilateral entorhinal thickness. For the in-house cohort used in local adaptation, only age, gender, and MRI-derived segmentation volumes were available and used.

The output is a synthetic FDG-PET volume intended to resemble a real clinical PET scan. The paper also reports visualization with 3D-SSP maps to display disease-specific hypometabolic patterns. These outputs are used in two downstream settings: human diagnostic reading and automated classification. This suggests that the framework is intended not merely for image synthesis, but for preserving diagnostically actionable structure in the generated functional images.

2. Diffusion-bridge formulation

The central methodological choice in SiM2P is a denoising diffusion bridge model (DDBM) adapted to 3D medical imaging (Li et al., 17 Oct 2025). The bridge formulation differs from unconditional or purely noise-to-image diffusion in that it treats MRI and PET as paired endpoints and constructs a stochastic process anchored to subject-specific anatomy.

The forward process is written as the continuous SDE

$$d \mathbf{x}_t = \mathbf{f}(\mathbf{x}_t, t)\, dt + g(t)\, d\mathbf{w}_t, \qquad \mathbf{x}_0 = \mathbf{x},$$

where $\mathbf{x}_0$ is the PET scan and $\mathbf{x}_T = \mathbf{y}$ is the corresponding MRI. To force the trajectory to hit the MRI endpoint at time $T$, the model applies a Doob $h$-transform and adds a guiding drift term:

$$d \mathbf{x}_t = \mathbf{f}(\mathbf{x}_t, t)\, dt + g(t)^2 \mathbf{h}(\mathbf{x}_t,t,\mathbf{y},T) + g(t)\, d\mathbf{w}_t,$$

with

$$\mathbf{h}(\mathbf{x}_t,t,\mathbf{y},T)=\nabla_{\mathbf{x}_t}\log p(\mathbf{x}_T \mid \mathbf{x}_t)\big|_{\mathbf{x}_T=\mathbf{y}}.$$

Generation uses the reverse-time process:

$$d\mathbf{x}_t = \left[ \mathbf{f}(\mathbf{x}_t, t) - g^2(t)\left(\mathbf{s}(\mathbf{x}_t,t,\mathbf{y},T)-\mathbf{h}(\mathbf{x}_t,t,\mathbf{y},T)\right) \right] dt + g(t)\, d\widehat{\mathbf{w}}_t,$$

where

$$\mathbf{s}(\mathbf{x}_t,t,\mathbf{y},T)=\nabla_{\mathbf{x}_t}\log q(\mathbf{x}_t \mid \mathbf{x}_T)\big|_{\mathbf{x}_T=\mathbf{y}}$$

is the true conditional score, approximated by a neural network.

This formulation gives SiM2P a probabilistic interpretation: rather than outputting a single fixed PET image from MRI, it models the conditional distribution of PET given MRI and covariates. A plausible implication is that this is particularly appropriate for dementia imaging, where metabolic phenotypes can be heterogeneous even among subjects with similar structural MRI findings.

3. Architecture, conditioning, and optimization

SiM2P uses a 3D diffusion Transformer backbone $F_\theta$ with an EDM-style parameterization (Li et al., 17 Oct 2025):

$\mathbf{x}_0$0

where $\mathbf{x}_0$1 denotes the auxiliary patient information. The learning objective is a weighted MSE against the clean PET:

$\mathbf{x}_0$2

with

$\mathbf{x}_0$3

In operational terms, the network reconstructs the original PET volume from noisy bridge states while being conditioned on the MRI endpoint and auxiliary covariates.

The Transformer is large-scale by medical-imaging standards. The $\mathbf{x}_0$4 volume is patchified into non-overlapping $\mathbf{x}_0$5 voxel patches, projected to embeddings using a learnable 3D convolution, augmented with frequency-based 3D sine-cosine positional embeddings, and processed through 28 Transformer blocks. Conditioning is injected through adaptive layer normalization zero (adaLN-Zero). The timestep embedding and auxiliary-data embedding are concatenated and used to modulate attention and feed-forward sublayers. Reported model scale is an embedding dimension of 1152, 16 attention heads, and about 904 million parameters.

Training uses Adam, learning rate 1e-4, no weight decay, batch size 1, and a single NVIDIA H100 GPU for 180k iterations, with evaluation every 10k iterations and checkpoint selection by best validation performance. Training is reported as approximately 96 GPU hours, whereas inference is much cheaper, at roughly 2 minutes per generated case under the chosen sampling regime. Sampling uses the higher-order hybrid solver from DDBM, based on a second-order Heun sampler plus scheduled Euler–Maruyama steps, with 100 sampling steps as the reported practical balance between quality and runtime.

The paper also reports several ablations. Removing auxiliary data worsened performance, increasing MAE from 0.0196 to 0.0212 and reducing PSNR from 28.62 to 27.72. Cognitive scores and ApoE4 were identified as especially informative among auxiliary variables. Adding explicit 3D-SSP supervision did not improve performance and slightly hurt MAE. A VP bridge outperformed a VE bridge, with VE reported at MAE 0.0274 and SSIM 0.727, versus VP at MAE 0.0196 and SSIM 0.939. Sampling quality improved with more steps, but gains diminished beyond about 100 steps.

4. Data basis and empirical performance

The training corpus comprised 1,860 paired MRI–FDG-PET subjects from ADNI (1,247 subjects), J-ADNI (319), and an in-house TUM cohort (323) (Li et al., 17 Oct 2025). The merged dataset included healthy controls, mild cognitive impairment, AD, and FTLD/bvFTD-related cases. MRI and PET were preprocessed to standard space, skull stripped, normalized, and resized to $\mathbf{x}_0$6 for modeling.

The clinical reader study focused on a held-out in-house set of 62 subjects: 22 cognitively normal controls, 19 AD, and 21 bvFTD. The study was blinded and involved four expert raters: 2 neuroradiologists reading MRI and 2 nuclear medicine physicians reading simulated PET. All had more than five years of experience and were blinded to labels, cohort distributions, and each other’s judgments. Each case was evaluated in two stages—dementia present versus absent, then AD versus bvFTD if dementia was present—and confidence was rated as low, moderate, or high. Confidence-weighted accuracy used weights 1, 2, and 3:

$\mathbf{x}_0$7

The core reader-study results are summarized below.

Task	SimPET	MRI
Dementia detection accuracy	92.74%	83.06%
AD vs bvFTD accuracy	84.68%	79.10%
Three-class CN vs AD vs bvFTD accuracy	84.68%	75.00%

For dementia detection, the absolute gain was 9.69 points, the relative gain was 11.7%, and the difference was statistically significant at $\mathbf{x}_0$8. For AD versus bvFTD, the gain was 4.70 points, but the unweighted comparison was reported as not clearly significant ($\mathbf{x}_0$9). For three-class diagnosis, the gain was 9.68 points with $\mathbf{x}_T = \mathbf{y}$0.

Confidence-weighted analysis amplified the benefit of SiM2P. Dementia detection reached 95.53% for SimPET versus 86.88% for MRI, with $\mathbf{x}_T = \mathbf{y}$1. AD versus bvFTD reached 92.22% versus 86.21%, with $\mathbf{x}_T = \mathbf{y}$2. Three-class diagnosis reached 90.94% versus 81.04%, also with $\mathbf{x}_T = \mathbf{y}$3. Interrater agreement likewise improved: for dementia detection, $\mathbf{x}_T = \mathbf{y}$4 versus MRI $\mathbf{x}_T = \mathbf{y}$5; for AD versus bvFTD, $\mathbf{x}_T = \mathbf{y}$6 versus $\mathbf{x}_T = \mathbf{y}$7; and for three-class diagnosis, $\mathbf{x}_T = \mathbf{y}$8 versus $\mathbf{x}_T = \mathbf{y}$9. The paper interprets these as evidence that the simulated PET images provide clearer disease signatures.

SiM2P was also compared with Pix2Pix, ResViT, BBDM, and PASTA. On the merged test set it achieved MAE $T$0, MSE $T$1, PSNR $T$2, and SSIM $T$3, outperforming the competing methods. In downstream automated three-class dementia classification using 3D ResNet-18, simulated PET from SiM2P nearly matched real PET, with macro AUC 0.919 versus real PET 0.972, and macro AP 0.874 versus real PET 0.940.

5. Local-Adapt and practical deployment

A major translational element of the framework is Local-Adapt, a two-stage workflow consisting of pretraining on large public datasets followed by fine-tuning on a small local cohort to match scanner, protocol, and site characteristics (Li et al., 17 Oct 2025). The paper emphasizes that local deployment does not require the full auxiliary set used in pretraining. Fine-tuning can be performed with age, gender, and MRI-derived segmentation volumes alone.

The study explicitly tested local-data reductions to 50%, 33%, 20%, and 10%, with 10% corresponding to 19 cases. Reported performance remained robust, with no significant degradation in key metrics across these small-data settings. The authors therefore argue that ~20 local cases may be sufficient for practical adaptation. Pretraining is said to take a few days, whereas fine-tuning requires only hours.

This deployment strategy is important because the paper frames generalizability partly as a site-adaptation problem rather than only as a model-scale problem. A plausible implication is that the Local-Adapt workflow is intended to compensate for domain shift introduced by local MRI and PET acquisition differences without requiring a full institutional PET-scale dataset.

The code is reported as publicly available at https://github.com/Yiiitong/SiM2P.

6. Interpretation, failure modes, and limitations

The paper’s qualitative analysis distinguishes between success cases and failure cases (Li et al., 17 Oct 2025). In success cases, MRI showed only subtle atrophy and was misread as control, whereas SimPET reproduced temporoparietal hypometabolism and supported correct AD diagnosis. In failure cases, errors arose mainly in overlapping syndromes, especially frontal-variant AD versus bvFTD. The authors note that in such cases SimPET often reproduced the real PET pattern faithfully, yet the disease itself remained intrinsically difficult to separate. This suggests that some observed errors reflect biological overlap rather than only model deficiency.

The limitations are stated explicitly. The work was validated on the reported cohort but still requires prospective multi-center studies for generalizability. 3D diffusion bridge training is computationally expensive. Absolute biomarker quantification from simulated PET still requires validation. Clinical interpretation workflow also matters: nuclear medicine physicians are described as the ideal readers, while neuroradiologists may be more practical if PET specialists are unavailable. The paper further notes that deployment may require telemedicine infrastructure if expert review is centralized.

Future extensions proposed in the paper include application to other neurodegenerative disorders, other PET tracers such as amyloid or tau, and multimodal conditioning with CSF biomarkers including p-tau and t-tau. Within the scope actually demonstrated, however, SiM2P is defined by a specific claim: it is a 3D conditional diffusion-bridge model that generates diagnostic-quality FDG-PET from MRI plus auxiliary patient information and, in blinded evaluation, improves diagnostic accuracy, confidence, and interrater agreement relative to MRI alone (Li et al., 17 Oct 2025).