ViTranZheimer: Transformer-Based AD MRI Diagnosis

• ViTranZheimer is a transformer-based framework that models 3D brain MRI as video input to capture both local and long-range dependencies for accurate Alzheimer’s diagnosis.
• It integrates tubelet embedding with 12 Transformer encoder layers to achieve state-of-the-art multi-class accuracy and robust sensitivity/specificity.
• The method uses a detailed preprocessing pipeline and rigorous cross-validation on ADNI data to outperform conventional CNN and hybrid models by 1–2%.

ViTranZheimer refers to a transformer-based deep learning framework for automated Alzheimer’s disease (AD) diagnosis from structural 3D brain MRI, as introduced in "Leveraging Video Vision Transformer for Alzheimer’s Disease Diagnosis from 3D Brain MRI" (Akan et al., 27 Jan 2025). The method models volumetric MR images as video-like inputs to exploit both local and long-range dependencies among brain slices. ViTranZheimer achieves state-of-the-art multi-class accuracy and near-perfect sensitivity/specificity for normal controls (NC), mild cognitive impairment (MCI), and AD, setting a new performance standard for T1-weighted MRI-based neurodegeneration detection.

1. Input Modalities and Preprocessing Pipeline

ViTranZheimer ingests T1-weighted, skull-stripped 3D MRI volumes, specifically the 32 central coronal slices per subject (shape: $T\times H\times W\times 1$, $T=32$, $H=W=64$). Preprocessing consists of tissue segmentation (CAT12; GM/WM/CSF), skull-stripping, and spatial normalization to MNI-152 template space with SPM12. Only the central 32 slices, containing the bulk of cerebral anatomy, are retained; image re-sampling ensures all voxels share equal grid size ($64\times64\,$, no further resizing). No data augmentation is reported.

2. Model Architecture: Video Vision Transformer (ViViT) Core

ViTranZheimer models the 32-slice input volume as a video sequence using a ViViT backbone. The processing flow is:

2.1 Tubelet Embedding

• The 3D input is partitioned into $P_t \times P_h \times P_w$ tubelets. Configured here as $P_t=32$, $P_h=P_w=16$, yielding $N = (T/P_t) \cdot (H/P_h) \cdot (W/P_w) = 16$ tokens.
• Each tubelet $X_i$ is flattened and embedded as $z_i^0 = W_E X_i + b_E$, $T=32$0, where $T=32$1 is the embedding dimension. Positional encoding $T=32$2 (plus a special CLS token) is added to form $T=32$3.

2.2 Transformer Encoder Blocks

• The backbone comprises $T=32$4 Transformer encoder layers, each containing LayerNorm, multi-head self-attention (MHSA, $T=32$5 heads), residual connections, and a position-wise feed-forward network (FFN).
• For an input $T=32$6:

$T=32$7

$T=32$8

• MHSA is realized as:

$T=32$9

with each head computed by

$H=W=64$0

2.3 Classification Head

• The $H=W=64$1-th layer's CLS embedding, $H=W=64$2, is input to a softmax classifier:

$H=W=64$3

yielding a three-class probability over {NC, MCI, AD}.

3. Training Procedure

Optimization proceeds via categorical cross-entropy:

$H=W=64$4

Model parameters are minimized using Adam ($H=W=64$5, $H=W=64$6, $H=W=64$7), learning rate $H=W=64$8, batch size 128, up to 1500 epochs. Early stopping checkpoints are triggered by validation loss improvement. No explicit dropout or weight decay is used; regularization relies on early stopping.

4. Benchmarking and Evaluation

4.1 Dataset Splitting and Protocol

The study employs the ADNI1 "3Yr 3T" dataset with 351 MRI volumes (75 NC, 156 MCI, 120 AD). Standard splits are 60%/20%/20% for training/validation/test and a repeated 10-fold stratified cross-validation.

4.2 Performance Metrics

Standard metrics are computed:

• Accuracy: $H=W=64$9
• Precision: $64\times64\,$0
• Recall (Sensitivity): $64\times64\,$1
• F1-score: $64\times64\,$2
• Specificity: $64\times64\,$3

4.3 Quantitative Results

Mean (±std) metrics over cross-validation:

Model	Accuracy (%)	Precision	Recall	F1
CNN-BiLSTM	96.479 ±2.205	0.96	0.96	0.96
ViT-BiLSTM	97.465 ±2.164	0.97	0.97	0.97
ViTranZheimer	98.6 ±1.4	0.97	0.97	0.97

ViTranZheimer achieves 98.6% accuracy, outperforming prior slice/voxel-based CNN or hybrid ViT-RNN frameworks on similar ADNI subsets.

Class-level sensitivity/specificity for ViTranZheimer:

• NC: 100% / 100%
• MCI: 98% / 99%
• AD: 97% / 100%

The improvement of 1–2 percentage points over baselines exceeds the reported standard deviations, supporting statistical robustness.

5. Self-Attention Visualization and Interpretability

While explicit heatmaps are not presented in (Akan et al., 27 Jan 2025), self-attention from the final CLS-to-token layers can be back-projected to 3D space to localize discriminative regions. Regions with elevated attention typically include hippocampal, medial temporal, and ventricular structures—areas associated with early AD pathology. Such back-projection supports anatomical relevance and offers potential interpretability.

6. Comparative Strengths, Limitations, and Prospects

Key technical advantages of ViTranZheimer:

• End-to-end learning of spatio-temporal dependencies in 3D volumetric MRI, eliminating the need for decoupled 2D feature extraction plus sequential modeling.
• Factorized self-attention captures intra-slice and inter-slice dependencies simultaneously.
• The model possesses a compact parameter footprint (466 K) compared to typical 3D-CNNs, which favors stability and sample-efficiency for moderate dataset sizes.
• Superior performance to hybrid ViT+BiLSTM or CNN+BiLSTM alternatives: direct optimization of tubelet embedding and classification, elimination of LSTM’s sequential bias, and increased sensitivity to complex volumetric degeneration patterns.

Reported limitations:

• No explicit data augmentation or harmonization for acquisition or site differences.
• External validation across other scanner types or lower-field images is not included.
• Interpretability analysis remains limited to potential attention map projection; further studies of model rationales are suggested.
• Generalization to prodromal phases (e.g., subjective cognitive decline) and longitudinal prediction of MCI→AD conversion remain open.

ViTranZheimer, as demonstrated on ADNI data, establishes video vision transformers with tubelet tokenization and pure self-attention as a new state-of-the-art for multi-class AD diagnosis using 3D MRI, combining minimal inductive bias, high parametric efficiency, and robust empirical performance (Akan et al., 27 Jan 2025).