DINO-BOLDNet: Synthesizing BOLD from T1w MRI

• The paper introduces DINO-BOLDNet, which directly synthesizes mean BOLD fMRI from T1w images using a frozen DINOv3 transformer, outperforming conditional GAN baselines.
• It employs a slice-wise attention fusion module and multi-scale generation decoder to integrate axial context and enhance anatomical fidelity in the reconstructed BOLD images.
• The framework uses advanced loss functions—including DINO-based perceptual, masked L1, MS-SSIM, and gradient losses—to ensure precise structural-to-functional mapping and improved image quality.

DINO-BOLDNet is a framework for synthesizing mean BOLD (Blood Oxygen Level Dependent) functional MRI images directly from T1-weighted (T1w) structural scans. By leveraging a frozen, self-supervised transformer encoder (DINOv3 ViT-B/16), slice-wise axial context fusion, and a multi-scale generation decoder, DINO-BOLDNet addresses the problem of BOLD image recovery in scenarios where functional scans are corrupted or unavailable. Experimental results on a clinically curated cohort demonstrate its superiority over conditional GAN baselines, both quantitatively and qualitatively, for the task of structural-to-functional MRI image mapping (Wang et al., 9 Dec 2025).

1. Network Architecture

1.1 Frozen DINOv3 Structural Encoder

DINO-BOLDNet treats the 3D T1w input $X \in \mathbb{R}^{H \times W \times Z}$ as overlapping $K=5$-slice axial windows: $$W_z = \bigl\{\,X_{z - 2}, X_{z - 1}, X_z, X_{z + 1}, X_{z + 2}\bigr\}.$$ Each slice is replicated to three channels and resampled to $224 \times 224$, yielding an input of shape $(B, K, 3, 224, 224)$. This is processed by a pretrained, frozen DINOv3 ViT-B/16. Each slice yields $L=12$ transformer block outputs, resulting in token matrices

$T_k^{(L)} \in \mathbb{R}^{196 \times 768},$

with $196$ patch tokens covering a $14 \times 14$ grid, stacked as

$T^{(L)} \in \mathbb{R}^{K \times 196 \times 768}.$

Additionally, skip connections are extracted at layers $K=5$0: $K=5$1 All DINOv3 weights are kept frozen, avoiding re-training or fine-tuning.

1.2 Slice-Wise Attention Fusion Module

To enforce contextually informed anatomical representation, self-attention is applied along the slice dimension. For stacked tokens $K=5$2 ($K=5$3, $K=5$4) for each patch, multi-head attention is computed over the axial slices: $K=5$5 where $K=5$6, $K=5$7, $K=5$8, $K=5$9, $$W_z = \bigl\{\,X_{z - 2}, X_{z - 1}, X_z, X_{z + 1}, X_{z + 2}\bigr\}.$$0 heads. For each patch position $$W_z = \bigl\{\,X_{z - 2}, X_{z - 1}, X_z, X_{z + 1}, X_{z + 2}\bigr\}.$$1: $$W_z = \bigl\{\,X_{z - 2}, X_{z - 1}, X_z, X_{z + 1}, X_{z + 2}\bigr\}.$$2

$$W_z = \bigl\{\,X_{z - 2}, X_{z - 1}, X_z, X_{z + 1}, X_{z + 2}\bigr\}.$$3

with final fused tokens $$W_z = \bigl\{\,X_{z - 2}, X_{z - 1}, X_z, X_{z + 1}, X_{z + 2}\bigr\}.$$4. Identical attention is applied to all skip tensors $$W_z = \bigl\{\,X_{z - 2}, X_{z - 1}, X_z, X_{z + 1}, X_{z + 2}\bigr\}.$$5 to yield $$W_z = \bigl\{\,X_{z - 2}, X_{z - 1}, X_z, X_{z + 1}, X_{z + 2}\bigr\}.$$6.

1.3 Multi-Scale Generation Decoder

The main-branch fused tokens $$W_z = \bigl\{\,X_{z - 2}, X_{z - 1}, X_z, X_{z + 1}, X_{z + 2}\bigr\}.$$7 are reshaped to $$W_z = \bigl\{\,X_{z - 2}, X_{z - 1}, X_z, X_{z + 1}, X_{z + 2}\bigr\}.$$8. Decoding stages $$W_z = \bigl\{\,X_{z - 2}, X_{z - 1}, X_z, X_{z + 1}, X_{z + 2}\bigr\}.$$9 repeatedly bilinearly upsample $224 \times 224$0 spatially, concatenate the resized skip tensor $224 \times 224$1, and apply $224 \times 224$2 convolution, GroupNorm, and GELU. A final $224 \times 224$3 convolution reduces channels to one, and the output is inverse-resampled under mask $224 \times 224$4 to recover the predicted mean BOLD image $224 \times 224$5 per slice, reassembled into 3D volume.

2. Training Objectives

2.1 DINO-Based Perceptual Loss

Perceptual alignment in DINOv3’s transformer feature space is enforced by

$224 \times 224$6

with $224 \times 224$7 denoting frozen DINOv3 features at layers $224 \times 224$8.

2.2 Additional Losses

• Masked L1: Voxel-wise fidelity in the masked brain region,

$224 \times 224$9

where $(B, K, 3, 224, 224)$0 is the ground truth mean BOLD.

• MS-SSIM Loss: Promotes perceptual-structural similarity,

$(B, K, 3, 224, 224)$1

• Gradient Loss: Preserves spatial edges,

$(B, K, 3, 224, 224)$2

The total loss is a convex sum: $(B, K, 3, 224, 224)$3 with weights $(B, K, 3, 224, 224)$4.

3. Dataset, Preprocessing, and Hyperparameters

• Subjects ($(B, K, 3, 224, 224)$5): Alzheimer’s disease (30), mild cognitive impairment (70), subjective cognitive decline (56), cognitively normal (92).
• T1w: 3D-MPRAGE; BOLD: GRE-EPI acquired on the same 3T scanner.
• Preprocessing via DeepPrep: T1w (bias correction, skull-strip, registration), BOLD (motion, slice-timing, distortion correction, MNI152 registration).
• Mean BOLD computed by discarding first 10 time volumes and averaging remainder.
• Data split: 80% train, 20% validation. All volumes resampled to MNI152 space at matched voxel size.

Optimization:

• AdamW (weight decay $(B, K, 3, 224, 224)$6).
• Initial learning rate $(B, K, 3, 224, 224)$7, cosine annealing to $(B, K, 3, 224, 224)$8.
• Batch size: 32, 100 epochs, RTX 3090 GPU.

Model specifics:

• DINOv3 ViT-B/16 encoder frozen, slice-attention heads = 4, attention layers = 2, decoder base channels = 512.

4. Quantitative and Qualitative Performance

4.1 Quantitative Comparison

Model	PSNR (dB)	MS-SSIM
cGAN baseline	30.41	0.9588
DINO-BOLDNet	31.10	0.9613

DINO-BOLDNet achieves an absolute PSNR gain of approximately 0.69 dB and a MS-SSIM gain of roughly 0.0025 over the cGAN baseline, with improvements consistent across the 20% validation cohort (no p-values reported) (Wang et al., 9 Dec 2025).

4.2 Qualitative Assessment

Visual comparison (Figure 1 in (Wang et al., 9 Dec 2025)) demonstrates:

• Sharper anatomical boundaries (notably in cortical ribbon and ventricles).
• Reduced error map residuals, particularly in high-contrast sulcal regions.

5. Interpretability, Advantages, and Applications

5.1 Rationale for Approach

• Self-supervised transformer (DINOv3) guidance: Encodes distributed, global anatomical priors, surpassing data-hungry CNNs in medical settings.
• Frozen encoder: Prevents overfitting, leveraging robust pretraining.
• Multi-slice attention: Explicitly models 3D anatomical dependencies critical for accurate structure-to-function translation.
• DINO-based perceptual loss: Aligns high-level texture and structure, increasing anatomical plausibility of synthesized BOLD.

5.2 Prospective Impact and Use Cases

• Missing/corrupted BOLD recovery: Enables downstream fMRI analysis where BOLD data is missing or unusable.
• Retrospective cohort enrichment: Empowers functional analysis in T1-only legacy datasets.
• Biomarker augmentation: Infers functional correlates to supplement structural diagnosis, e.g., in Alzheimer’s disease.
• Subject-specific functional atlases: Supports neurosurgical planning and personalized connectomic investigation.

DINO-BOLDNet is the first reported architecture capable of directly generating mean BOLD MRI from T1w data, demonstrating improved fidelity over cGAN baselines and promising broader applicability in research and clinical environments (Wang et al., 9 Dec 2025).