MethConvTransformer for AD Methylation Analysis

• MethConvTransformer is a deep learning framework that combines CpG-level linear projections, convolutional extraction, and transformers for robust Alzheimer’s detection.
• The model employs multi-head self-attention and explicit CpG attributions to achieve state-of-the-art cross-tissue performance in AD prediction.
• Its integrated interpretability tools, including SHAP values and Grad-CAM++, enable detailed biomarker discovery and mechanistic insights in neurodegenerative research.

MethConvTransformer is a transformer-based deep learning framework specifically designed for robust, cross-tissue detection of Alzheimer’s disease (AD) from DNA methylation data. It integrates per-CpG linear projections, convolutional feature extraction, multi-head self-attention, and context embeddings to jointly capture local and long-range dependency structures in methylomic profiles, while incorporating biological covariates and tissue information. The architecture provides explicit CpG-level attributions, seamless multi-tissue generalization, and achieves state-of-the-art discrimination in cross-tissue AD prediction tasks. MethConvTransformer delivers both discrimination and multi-resolution interpretability, supporting epigenetic biomarker discovery and mechanistic hypothesis generation in neurodegenerative disease research (Qu et al., 1 Jan 2026).

1. Architectural Design

MethConvTransformer processes methylation profiles $x_i\in\mathbb{R}^P$ for %%%%1%%%% CpG sites per subject $i$. The input stage is a CpG-wise linear projection: $h_{i,j} = w_j\, x_{i,j} + b_j, \quad j=1,\dots,P$ where $w\in\mathbb{R}^P$ and $b\in\mathbb{R}^P$ are learned per-CpG parameters, yielding a vector $h_i\in\mathbb{R}^P$ termed the margin map. This mapping renders each CpG effect size explicit and makes the transformation directly interpretable.

The margin map is processed by a stack of one-dimensional convolutional layers applied along the CpG axis, typically:

• Conv$_1$: kernel size=$3$, filters=$64$, stride=$2$ ($$\rightarrow \mathbb{R}^{\lfloor P/2 \rfloor \times 64}$$)
• Conv$_2$: kernel size=$3$, filters=$d$, stride=$2$ ($\rightarrow \mathbb{R}^{L\times d}$, with $L\approx P/4$)

Convolutions are followed by ReLU activations and optional pooling for local feature encoding and dimensionality reduction.

The output sequence $T_i\in\mathbb{R}^{L\times d}$ is passed through $L_T$ stacked transformer blocks, each with multi-head self-attention and position-wise feed-forward sublayers. For every block $\ell$: $$\begin{align*} H_i^{(0)} &= T_i\ H_i^{(\ell+1)} &= \mathrm{TransformerBlock}^{(\ell)}\left(H_i^{(\ell)}\right) \end{align*}$$ Each block’s attention sublayer follows the canonical scheme: $$\begin{align*} Q^{(h)} &= HW_Q^{(h)},\qquad K^{(h)} = HW_K^{(h)},\qquad V^{(h)} = HW_V^{(h)}\ A^{(h)} &= \mathrm{softmax}\left( \frac{Q^{(h)} (K^{(h)})^\top}{\sqrt{d_h}} \right) \end{align*}$$ with $H$ heads, producing re-mapped sequence representations at each layer.

After transformer encoding, token features are mean-pooled across positions to yield a vector $\,\hat u_i\in\mathbb{R}^D$. This is concatenated with embeddings $E_{\text{cov}}(z_i)$ for subject-level covariates $z_i\in\mathbb{R}^K$ (age, sex, etc.), and $E_{\text{tiss}}(r_i)$ for tissue/region label $r_i\in\{1,\dots,R\}$: $$u_i = \hat u_i \,\|\, E_{\text{cov}}(z_i) \,\|\, E_{\text{tiss}}(r_i) \in \mathbb{R}^{D+d_{\text{cov}}+d_{\text{tiss}}}$$ The final head is a linear classifier with softmax over the output vector: $$\hat{y}_i = \mathrm{softmax}\left(W_c u_i + b_c\right)$$

2. Training Methodology and Preprocessing

MethConvTransformer is trained end-to-end on preprocessed methylation matrices derived from raw Illumina IDATs or $\beta$-matrices (after ChAMP pipeline: probe filtering, BMIQ normalization, and ComBat batch correction across studies). Clinical covariates are z-scored or integer-encoded, and feature-selection is performed per tissue by variance, with a typical union size $P\approx 10^4$–$1.5 \times 10^4$ CpGs.

The compound training loss combines label-smoothed cross-entropy (with smoothing parameter $\epsilon$) and a CpG-wise margin regularizer: $$\mathcal{L} = \frac{1}{|\mathcal{B}|}\sum_{i\in\mathcal{B}}\left[ \mathcal{L}_{\mathrm{CE}}(\hat y_i, \tilde{y}_i) + \alpha\, \mathcal{R}_{\mathrm{margin}}(y^{\pm1}_i, m_i) \right]$$ where $\tilde{y}$ are smoothed labels, and

$$\mathcal{R}_{\mathrm{margin}} = \frac{1}{k}\sum_{t\in\text{Top-}k}\max\big[0, 1 - y_i^{\pm1} m_{i,t}\big]$$

penalizes the $k$ hardest-misclassified CpGs. Optimization is via Adam with decoupled weight decay; learning rate, batch size (typically 32), epochs, and architecture hyperparameters are selected by Optuna’s TPE sampler and early stopping.

3. Evaluation Benchmarks

MethConvTransformer was benchmarked on six GEO datasets and an ADNI blood cohort representing a total of 1,656 samples (908 AD/748 CN) over ten brain and peripheral tissues. Performance metrics include AUC, accuracy, and F1-score, averaged across ten random seeds. Summary results are as follows:

Dataset	AUC	Accuracy	F1-score
ADNI (blood)	0.55 ± 0.07	0.62 ± 0.02	0.37 ± 0.14
GSE125895 (cortex+CB)	0.95 ± 0.04	0.91 ± 0.09	0.79 ± 0.29
GSE134379 (MTG & CB)	0.62 ± 0.04	0.63 ± 0.03	0.69 ± 0.05
GSE66351 (neurons/glia)	0.74 ± 0.11	0.78 ± 0.07	0.84 ± 0.05
GSE59685 (multi-tissue)	0.95 ± 0.09	0.94 ± 0.07	0.96 ± 0.05
GSE80970 (PFC & STG)	0.90 ± 0.08	0.85 ± 0.12	0.86 ± 0.10
GSE144858 (blood)	0.66 ± 0.11	0.69 ± 0.06	0.58 ± 0.15
Combined (cross-tissue)	0.842 ± 0.021	0.774 ± 0.022	0.803 ± 0.017

Compared against baselines including GaussianNB, KNN, LDA, SVM, logistic regression (L1/L2), RandomForest, and GradientBoosting, MethConvTransformer achieved the highest or statistically indistinguishable AUC; Welch's $t$-test supported significant improvement over most baselines ($p<0.05$) (Qu et al., 1 Jan 2026).

4. Interpretability Approaches

MethConvTransformer supports multi-layered interpretability.

• Linear Projection Weights: The learned per-CpG weights $w$ from the initial linear layer directly quantify the contribution of each CpG to the margin; their values can be interpreted as effect sizes.
• SHAP Values: Per-sample, per-CpG Shapley values $\phi_j^{(c)}$ decompose the model output into additive feature attributions, illuminating variable and directionality of site effects.
• Grad-CAM++: Applied post-convolution, this yields regionally-resolved saliency maps $M^c(t)$ that highlight CpG blocks with highest relevance for each predicted class.
• Transformer Attention Maps: Aggregating self-attention weights $\bar{A}$ from transformer heads enables visualization of long-range non-linear methylation dependencies between CpG sets.

Multi-resolution interpretability links single-site effects, local blocks, and global interaction patterns to biological context.

5. Biological Insights and Pathway Enrichment

Model-driven interpretability analyses converge on sparse, cluster-forming methylation signatures for AD, centered on key CpG loci in cerebellum and temporal cortex, with blood showing lower but non-trivial signal. Enrichment analyses on the highest-magnitude $w_j$ and SHAP values reveal over-representation in the following pathways:

• Immune receptor signaling: including activation of immune responses and tyrosine kinase activity.
• Glycan and mucin-type O-glycosylation: predominantly O-glycan biosynthesis.
• Lipid metabolism and Golgi organization: glycosphingolipid metabolism, Golgi cisterna/stack, vesicular trafficking, and energy production.
• ER/Golgi stress and related comorbidities: hydrolase activity, GPCR signaling, endoplasmic reticulum organization, type I diabetes, and viral carcinogenesis.

This supports mechanistic links between AD-related neuroinflammation, glycosylation dysregulation, lipid metabolism defects, and endomembrane stress (Qu et al., 1 Jan 2026).

6. Significance and Implications

MethConvTransformer demonstrates that transformer-based frameworks with explicit CpG-wise linear attribution and dedicated convolutional feature encoding provide a principled approach to cross-tissue DNA methylation analysis. The model achieves state-of-the-art AD discrimination, surpassing or matching all conventional machine learning baselines in cross-tissue benchmarks and producing interpretable, testable biological insights. This suggests transformer architectures with margin-aware loss and feature-level transparency can bridge discovery and translational applications in epigenomics. The compound loss balances classification and feature selectivity, supporting sparse and robust biomarker identification. A plausible implication is improved reproducibility and mechanistic grounding for methylation-based disease diagnostics.