CALM: Interpretable Cross-Modal Alignment for Biomarker Discovery from Unpaired Data

Abstract: The interaction between brain structure and genetic influences is key to understanding neuropsychiatric disorders. However, most large-scale datasets are unimodal, providing either neuroimaging or genetics data. We propose CALM, a framework that learns interpretable associations between brain ROIs and genetic pathways from completely disjoint populations. CALM aligns the two modalities in a shared latent space via linear projections that simultaneously match the class-conditional latent distributions and ensure group separability. These projections provide interpretable pathway--ROI associations. When trained on unimodal imaging and genetics datasets, CALM generalizes to an unseen paired dataset, outperforming several state-of-the-art methods and ablation baselines. We also demonstrate stability of the learned associations against a paired baseline. Our experiments on autism spectrum disorder reveal immune and metabolic pathways linked to specific cortical regions and are consistent with established literature. Thus, CALM opens the door to leveraging large unimodal repositories for studying cross-modal interactions in brain disorders across disparate datasets.

• The paper demonstrates that cross-modal alignment from unpaired imaging and genetics data can robustly recover biologically relevant pathway–ROI associations.
• It employs modality-specific encoders and learnable linear projections to map features into a shared latent space using class-conditional MMD and contrastive losses.
• CALM achieves competitive ASD classification performance while providing interpretable associations that guide scalable biomarker discovery.

Interpretable Cross-Modal Alignment of Neuroimaging and Genetics with CALM

Introduction and Motivation

Neuropsychiatric disorders such as autism spectrum disorder (ASD) are governed by intricate interactions between brain morphology and genetic architecture. High-quality, large-scale datasets are predominantly unimodal, presenting a major barrier to integrative analyses required for biomarker discovery. Most previous multimodal learning paradigms, especially in imaging-genetics, presuppose paired datasets that contain both neuroimaging and genetic information from the same subjects—a requirement seldom satisfied at scale due to practical and economic constraints.

CALM (Class-conditional Alignment with Linear Maps) addresses this critical limitation by enabling interpretable cross-modal association discovery between unimodal, unpaired datasets, thus maximizing use of the largest available neuroimaging and genetics repositories for ASD. The framework learns to project latent representations of both modalities into a shared latent space to reveal pathway–region-of-interest (ROI) associations without requiring overlapping subject pools.

Figure 1: Overview of CALM, illustrating modality-specific encoders and linear projections aligning unpaired MRI and genetics data in a shared latent space. The learned association matrix quantifies pathway–ROI associations.

Methodology

Latent Representation Encoding

CALM operates via modality-specific pretrained encoders: an autoencoder and classifier for each modality (structural MRI and genetics). The encoding preserves entity-level granularity—region-level for imaging and pathway-level for genetics—retaining interpretability by ensuring latent representation rows correspond directly to neurobiological features.

Class-Conditional Linear Alignment

The core alignment is achieved via learnable linear projections, parameterized by matrices $\mathbf{W}_I$ (imaging) and $\mathbf{W}_G$ (genetics), mapping each modality’s representation into a shared $d$-dimensional latent space. The cross-modal association matrix $\mathbf{A} = \mathbf{W}_I^\top \mathbf{W}_G$ emerges as an explicit ROI–pathway association, interpretable in terms of latent space collinearity.

Alignment in the shared space is enforced through a compound objective:

• Class-Conditional MMD Loss ($\mathcal{L}_{\text{CMMD}}$): Aligns distributions within diagnostic groups, preventing negative transfer and maintaining semantic fidelity.
• Supervised Contrastive Loss ($\mathcal{L}_{\text{con}}$): Promotes group separability by clustering same-class latent representations, regardless of modality.
• Orthogonality Regularization ($\mathcal{L}_{\text{orth}}$): Prevents collapse in projection space, ensuring disentangled latent factors.

This construction bypasses the need for paired samples, in contrast to CCA-based and OT-based cross-modal methods, rendering CALM applicable to any domains with disjoint modality collections.

Experimental Setup

Datasets and Feature Extraction

Training utilized T1-weighted MRI from ABIDE I/II ($N = 1,898$) and genotypes from the Simons Simplex Collection (SSC, $N = 2,452$). The independent ACE cohort ($N = 198$) with paired imaging/genetics was reserved for evaluation. Neuroimaging features consisted of 246 ROI-wise morphometric measures (volume, area, thickness, and SD), and genetics input comprised pathway-level polygenic risk scores aggregated via KEGG.

Training Protocol

Encoders were pretrained before cross-modal alignment, after which linear projectors were optimized with the combined loss. Diagnostic prediction was evaluated by combining output probabilities from both classifiers on paired test data.

Baseline comparisons included G-MIND, UNSEEN, SUE, an optimal transport variant, and CALM ablations, all harmonized for feature processing.

Results

Classification Performance

On the held-out ACE dataset, CALM demonstrated the highest predictive performance (Accuracy $\mathbf{W}_G$0; AUC $\mathbf{W}_G$1), with improvements over baselines and ablations not crossing significance thresholds for the closest contenders. These results highlight the utility of the class-conditional, contrastively-regularized alignment formulation, especially relative to spectral or summation-based fusion strategies that do not preserve interpretable associations.

Validation of Pathway–ROI Associations

The key assertion of CALM is that interpretable cross-modal correlations are recoverable from entirely unpaired data. To test this, the linear association matrix $\mathbf{W}_G$2 learned in the unpaired setup was correlated with its analogue from a paired baseline using InfoNCE for direct subjectwise alignment.

Figure 2: (a) Strong Pearson correlation ($\mathbf{W}_G$3) between mean association matrices learned from unpaired and paired settings, supporting recovery of biologically relevant associations from unpaired data. (b) Stability of associations across folds measured by SNR.

A high inter-method correlation ($\mathbf{W}_G$4) substantiates CALM's ability to learn robust, generalizable pathway–ROI mappings absent paired training data. Signal-to-noise analysis further indicated that 72% of associations were reproducible ($\mathbf{W}_G$5) across validation folds.

Interpretability of Discovered Associations

SALM identifies convergent pathways and brain regions reminiscent of ASD biology:

Figure 3: (a) Bipartite graph of top pathway–ROI associations; line width denotes association strength. (b) Attribution of each GWAS disorder trait to the shared latent space.

Immune (IgA), metabolic (Wnt, sulfur metabolism), and inflammatory (NOD-like receptor) pathways each mapped onto temporo-occipital, parietal, or limbic ROIs, consistent with literature implicating multisensory integration, homeostasis, and socioemotional circuit dysfunction in ASD. Attribution analysis revealed that ASD, ADHD, educational attainment, and IQ-associated channels dominate shared latent variance, reflecting the heritable, pleiotropic architecture of neurodevelopmental disorders.

Practical and Theoretical Implications

CALM’s framework allows for rigorous biomarker discovery using the full expanse of available unimodal data, unlocking new potential for retrospective reanalysis of existing imaging and genetics collections. The explicit linear association matrix contrasts with deep fusion methods that obscure the mapping between input features, preserving interpretability crucial for hypothesis generation and mechanistic insights.

Theoretically, CALM demonstrates that robust cross-modal representation learning—traditionally presumed infeasible without paired data—can indeed recover biologically meaningful associations when conditioned on diagnostic groups. This aligns with the growing appreciation for class-conditional structure in multi-omics integration and domain adaptation.

Future Implications

Anticipated extensions of CALM involve adaptation to more complex, nonlinear association spaces, incorporation of additional phenotypes or cross-cohort harmonization, and transfer to other domains (e.g., transcriptomics–imaging). Additionally, the pathway–ROI mapping could guide downstream mechanistic studies, including causal modeling and intervention targeting in neurodevelopmental disorders.

Conclusion

CALM presents a class-conditional, interpretable alignment strategy for cross-modal biomarker discovery from unpaired neuroimaging and genetic datasets. It achieves state-of-the-art group-level prediction, demonstrates robustness of derived pathway–ROI associations, and delivers a scalable template for multi-cohort, multi-omic psychiatric research. The interpretability and statistical rigor of CALM address critical needs for translational and mechanistic research in complex neurogenetic disorders.