---
title: 'DINO-BOLDNet: Synthesizing BOLD from T1w MRI'
url: https://www.emergentmind.com/topics/dino-boldnet
type: topic
---

# DINO-BOLDNet: Synthesizing BOLD from T1w MRI

DINO-BOLDNet is a framework for synthesizing mean BOLD (Blood Oxygen Level Dependent) functional MRI images directly from T1-weighted (T1w) structural scans. By leveraging a frozen, self-supervised transformer encoder (DINOv3 ViT-B/16), slice-wise axial context fusion, and a multi-scale generation decoder, DINO-BOLDNet addresses the problem of BOLD image recovery in scenarios where functional scans are corrupted or unavailable. Experimental results on a clinically curated cohort demonstrate its superiority over conditional GAN baselines, both quantitatively and qualitatively, for the task of structural-to-functional MRI image mapping [2512.08337].

## 1. Network Architecture

### 1.1 Frozen DINOv3 Structural Encoder
DINO-BOLDNet treats the 3D T1w input $X \in \mathbb{R}^{H \times W \times Z}$ as overlapping $K=5$-slice axial windows:
\[
W_z = \bigl\{\,X_{z - 2}, X_{z - 1}, X_z, X_{z + 1}, X_{z + 2}\bigr\}.
\]
Each slice is replicated to three channels and resampled to $224 \times 224$, yielding an input of shape $(B, K, 3, 224, 224)$. This is processed by a pretrained, frozen DINOv3 ViT-B/16. Each slice yields $L=12$ transformer block outputs, resulting in token matrices
\[
T_k^{(L)} \in \mathbb{R}^{196 \times 768},
\]
with $196$ patch tokens covering a $14 \times 14$ grid, stacked as
\[
T^{(L)} \in \mathbb{R}^{K \times 196 \times 768}.
\]
Additionally, skip connections are extracted at layers $\ell \in \{3,6,9,12\}$:
\[
S^{(\ell)} \in \mathbb{R}^{K \times 196 \times 768}.
\]
All DINOv3 weights are kept frozen, avoiding re-training or fine-tuning.

### 1.2 Slice-Wise Attention Fusion Module
To enforce contextually informed anatomical representation, self-attention is applied along the slice dimension. For stacked tokens $T \in \mathbb{R}^{K \times N \times D}$ ($N=196$, $D=768$) for each patch, multi-head attention is computed over the axial slices:
\[
Q = T W_Q,\quad K = T W_K,\quad V = T W_V,
\]
where $W_Q$, $W_K$, $W_V \in \mathbb{R}^{D \times D_h}$, $D_h = D / H$, $H = 4$ heads. For each patch position $n=1, \dots, N$:
\[
A_n = \mathrm{softmax}\left(\frac{Q_n K_n^\top}{\sqrt{D_h}}\right) \in \mathbb{R}^{K \times K},
\]
\[
\widetilde T_n = A_n V_n \in \mathbb{R}^{K \times D_h},
\]
with final fused tokens $\widetilde T \in \mathbb{R}^{K \times N \times D}$. Identical attention is applied to all skip tensors $S^{(\ell)}$ to yield $\widetilde S^{(\ell)}$.

### 1.3 Multi-Scale Generation Decoder
The main-branch fused tokens $\widetilde T$ are reshaped to $F^{(0)} \in \mathbb{R}^{768 \times 14 \times 14}$. Decoding stages $j = 1,2,3,4$ repeatedly bilinearly upsample $F^{(j-1)}$ spatially, concatenate the resized skip tensor $\widetilde S^{(\ell_j)}$, and apply $3\times3$ convolution, GroupNorm, and GELU. A final $3\times3$ convolution reduces channels to one, and the output is inverse-resampled under mask $M$ to recover the predicted mean BOLD image $\hat Y$ per slice, reassembled into 3D volume. 

## 2. Training Objectives

### 2.1 DINO-Based Perceptual Loss
Perceptual alignment in DINOv3’s transformer feature space is enforced by
\[
\mathcal{L}_{\mathrm{perc}} = \sum_{\ell \in \{3,6\}} \left\lVert \phi_\ell(\hat Y) - \phi_\ell(Y) \right\rVert_1,
\]
with $\phi_\ell$ denoting frozen DINOv3 features at layers $\ell = 3,6$.

### 2.2 Additional Losses
- **Masked L1**: Voxel-wise fidelity in the masked brain region,
\[
\mathcal{L}_{\mathrm{L1}} = \left\lVert M \odot (\hat Y - Y) \right\rVert_1,
\]
where $Y$ is the ground truth mean BOLD.
- **MS-SSIM Loss**: Promotes perceptual-structural similarity,
\[
\mathcal{L}_{\mathrm{MS\text{-}SSIM}} = 1 - \mathrm{MS\text{-}SSIM}(M \odot \hat Y, M \odot Y).
\]
- **Gradient Loss**: Preserves spatial edges,
\[
\mathcal{L}_{\mathrm{grad}} = \left\lVert \nabla_x \hat Y - \nabla_x Y \right\rVert_1 + \left\lVert \nabla_y \hat Y - \nabla_y Y \right\rVert_1.
\]
The total loss is a convex sum:
\[
\mathcal{L} = \lambda_1\,\mathcal{L}_{\mathrm{L1}} + \lambda_s\,\mathcal{L}_{\mathrm{MS\text{-}SSIM}} + \lambda_g\,\mathcal{L}_{\mathrm{grad}} + \lambda_p\,\mathcal{L}_{\mathrm{perc}},
\]
with weights $\lambda_1=1.0,\,\lambda_s=0.5,\,\lambda_g=0.1,\,\lambda_p=0.05$.

## 3. Dataset, Preprocessing, and Hyperparameters

- Subjects ($n=248$): Alzheimer’s disease (30), mild cognitive impairment (70), subjective cognitive decline (56), cognitively normal (92).
- T1w: 3D-MPRAGE; BOLD: GRE-EPI acquired on the same 3T scanner.
- Preprocessing via DeepPrep: T1w (bias correction, skull-strip, registration), BOLD (motion, slice-timing, distortion correction, MNI152 registration).
- Mean BOLD computed by discarding first 10 time volumes and averaging remainder.
- Data split: 80% train, 20% validation. All volumes resampled to MNI152 space at matched voxel size.

**Optimization:**
- AdamW (weight decay $10^{-4}$).
- Initial learning rate $2\times10^{-4}$, cosine annealing to $10^{-6}$.
- Batch size: 32, 100 epochs, RTX 3090 GPU.
  
**Model specifics:**
- DINOv3 ViT-B/16 encoder frozen, slice-attention heads = 4, attention layers = 2, decoder base channels = 512.

## 4. Quantitative and Qualitative Performance

### 4.1 Quantitative Comparison

| Model         | PSNR (dB) | MS-SSIM   |
|---------------|-----------|-----------|
| cGAN baseline | 30.41     | 0.9588    |
| DINO-BOLDNet  | 31.10     | 0.9613    |

DINO-BOLDNet achieves an absolute PSNR gain of approximately 0.69 dB and a MS-SSIM gain of roughly 0.0025 over the cGAN baseline, with improvements consistent across the 20% validation cohort (no p-values reported) [2512.08337].

### 4.2 Qualitative Assessment
Visual comparison (Figure 4 in [2512.08337]) demonstrates:
- Sharper anatomical boundaries (notably in cortical ribbon and ventricles).
- Reduced error map residuals, particularly in high-contrast sulcal regions.

## 5. Interpretability, Advantages, and Applications

### 5.1 Rationale for Approach

- **Self-supervised transformer (DINOv3) guidance**: Encodes distributed, global anatomical priors, surpassing data-hungry CNNs in medical settings.
- **Frozen encoder**: Prevents overfitting, leveraging robust pretraining.
- **Multi-slice attention**: Explicitly models 3D anatomical dependencies critical for accurate structure-to-function translation.
- **DINO-based perceptual loss**: Aligns high-level texture and structure, increasing anatomical plausibility of synthesized BOLD.

### 5.2 Prospective Impact and Use Cases

- **Missing/corrupted BOLD recovery**: Enables downstream fMRI analysis where BOLD data is missing or unusable.
- **Retrospective cohort enrichment**: Empowers functional analysis in T1-only legacy datasets.
- **Biomarker augmentation**: Infers functional correlates to supplement structural diagnosis, e.g., in Alzheimer’s disease.
- **Subject-specific functional atlases**: Supports neurosurgical planning and personalized connectomic investigation.

DINO-BOLDNet is the first reported architecture capable of directly generating mean BOLD MRI from T1w data, demonstrating improved fidelity over cGAN baselines and promising broader applicability in research and clinical environments [2512.08337].

Source: https://www.emergentmind.com/topics/dino-boldnet