Multi-Modal Graph Neural Network with Transformer-Guided Adaptive Diffusion for Preclinical Alzheimer Classification

Abstract: The graphical representation of the brain offers critical insights into diagnosing and prognosing neurodegenerative disease via relationships between regions of interest (ROIs). Despite recent emergence of various Graph Neural Networks (GNNs) to effectively capture the relational information, there remain inherent limitations in interpreting the brain networks. Specifically, convolutional approaches ineffectively aggregate information from distant neighborhoods, while attention-based methods exhibit deficiencies in capturing node-centric information, particularly in retaining critical characteristics from pivotal nodes. These shortcomings reveal challenges for identifying disease-specific variation from diverse features from different modalities. In this regard, we propose an integrated framework guiding diffusion process at each node by a downstream transformer where both short- and long-range properties of graphs are aggregated via diffusion-kernel and multi-head attention respectively. We demonstrate the superiority of our model by improving performance of pre-clinical Alzheimer's disease (AD) classification with various modalities. Also, our model adeptly identifies key ROIs that are closely associated with the preclinical stages of AD, marking a significant potential for early diagnosis and prevision of the disease.

• The paper demonstrates that integrating modality-specific adaptive diffusion with transformer-guided attention significantly enhances preclinical Alzheimer classification and ROI interpretability.
• It employs dynamically learned heat kernel convolutions to capture local node features, effectively integrating heterogeneous neuroimaging biomarkers.
• The architecture outperforms standard GNN baselines while offering clinically valuable insights into region-specific brain network alterations in early Alzheimer’s disease.

Multi-Modal Graph Neural Network with Transformer-Guided Adaptive Diffusion for Preclinical Alzheimer Classification

Introduction and Motivation

Alzheimer’s disease (AD) disrupts neural connectomics through amyloid deposition and neurofibrillary tangles, manifesting as both structural and metabolic network aberrations in the early, preclinical stages. Conventional GNNs handling brain network data often rely on graph convolutional operators with local feature aggregation predicated on homophily, or deploy graph transformers for global attention. These paradigms exhibit inherent deficiencies: convolutional layers insufficiently capture distant node effects, while standard transformers lack node-centric interpretability necessary for clinical inference. Furthermore, multi-modal neuroimaging creates heterogeneous node feature spaces (e.g., MRI cortical thickness, PET $\beta$-Amyloid, FDG metabolic intensity), challenging aggregation and clinical translation.

The GTAD architecture addresses these limitations by integrating adaptive diffusion kernels with downstream transformer-guided attention, constructing a principled pipeline for both locality-sensitive and globally attentive feature learning. This enables robust classification and region-of-interest (ROI) interpretability for preclinical AD across modalities.

Figure 1: The GTAD workflow, showing modality-specific adaptive convolution followed by transformer-based attention and downstream classification.

Methodological Framework

GTAD employs a two-stage feature processing schema for multi-modal brain network graphs. The encoder constructs node embeddings via modality-specific heat kernel convolutions, where each kernel’s spatial scale is dynamically learned per ROI and modality (Equation~3). The diffusion kernel parameterization enables selective aggregation of local neighborhood information, crucially modulated by disease heterogeneity across neuroimaging biomarker modalities.

The transformer-guided attention block receives these embeddings, performing modality-wise multi-head self-attention over the concatenated feature spaces. This structure simultaneously integrates long-range dependencies and cross-modal information, with residual and layer normalization applied for stable deep context propagation (Equation~6). The classification head operates on the final, globally integrated representation, with loss gradients backpropagated to update kernel scales, thus tightly coupling local and global optimization for both predictive accuracy and ROI interpretability.

Experimental Validation

GTAD was benchmarked on a large preclinical ADNI cohort ($T$=919 subjects; CN, SMC, EMCI), with graph construction based on tractography between 160 ROIs and node features derived from three distinct neuroimaging biomarker modalities. Multi-class classification was performed on four feature combinations: cortical thickness with $\beta$-Amyloid, cortical thickness with FDG, $\beta$-Amyloid with FDG, and joint multi-modal integration.

GTAD exhibits superior performance compared to convolutional, diffusion-based, and transformer GNN baselines across all metrics (accuracy, precision, recall), consistently outperforming even advanced transformer-based architectures (DIFFormer, SGFormer) by +0.9% to +1.2% in accuracy and showing lower variance across folds.

Interpretability and ROI Analysis

The modality-wise scale parameters learned during training provide ROI-level insight into local neighborhood requirements for classification. Visualization of these node-wise scales across left/right hemispheres demonstrates substantial regional and modality-specific heterogeneity, with the smallest scales identifying ROIs where strictly local information is diagnostically salient (Figure 2). These regions include subcortical nuclei (putamen, thalamus, globus pallidus), temporal and frontal gyri, all known to be associated with early AD pathophysiology.

Figure 2: Spatial mapping of learned scales for cortical thickness, $\beta$-Amyloid, and FDG on left/right hemispheres. Highlighted ROIs with smallest scales reflect high local relevance for classification.

Transformer attention scores elucidate global inter-ROI influence. The distribution of modality-wise attention scores highlights ROIs with maximal cross-network influence, notably the lingual gyrus across all modalities, as well as hippocampus, putamen, and collat.transv.post in FDG, which aligns with established clinical findings on early metabolic and volumetric degradation in AD (Figure 3).

Figure 3: Attention score distributions per brain region for each modality, and corresponding ROIs with highest importance rates for classification.

Ablation and Architectural Insights

Ablation experiments reveal the critical effectiveness of modality-guided adaptive convolution, fundamentally exceeding standard graph convolution and MLP baselines. The incorporation of multi-modal versus position-wise attention further enhances accuracy by +1.8%, asserting the essentiality of both local and global selection mechanisms in heterogeneous graph modalities.

Implications and Future Directions

The formal coupling of adaptive local diffusion and globally contextual transformer attention in GTAD enables interpretable, modality-aware graph classification and supports identification of disease-specific ROIs critical for early detection. Practically, this methodology integrates multi-modal imaging in clinical workflows supporting precision diagnosis and stratification in preclinical AD, scalable to broader neurodegenerative disorders.

Theoretically, the framework advocates for joint optimization of local neighborhood scales and global transformer propagation in heterogeneous graphs, suggesting new directions for interpretable GNN design under complex multi-modal biomedical data. Future research will extend adaptive scale learning to dynamically integrate additional biomarkers and explore longitudinal modeling for disease trajectory prediction.

Conclusion

GTAD delivers a robust multi-modal GNN using transformer-guided adaptive diffusion for accurate classification and interpretability in preclinical AD network analysis. Its combination of node-wise modality-specific diffusion and global attention offers significant advances for both predictive performance and ROI interpretability, substantiating its utility for early-stage, multi-modal clinical diagnostic applications.