CASL-VAE: Learning Structured Latent Variables from Unpaired Data for Semi-supervised Clustering and Paired Sample Generation

Abstract: Quantifying variability in a target population relative to a reference population is central to many scientific and clinical problems (e.g., diseased vs. healthy). Yet, without paired data and in the presence of heterogeneous target variation, existing methods struggle to separate multiple modes of target-specific variation. We propose \textit{CASL-VAE}, a deep contrastive latent variable model that learns structured latent generative factors from unpaired data. CASL-VAE factorizes variation into continuous common latent factors shared across populations and hierarchical salient latent factors that model target-specific heterogeneity as discrete subtypes and continuous within-subtype variation. Using variational inference, we show how approximate joint likelihood optimization over reference and target domains can be performed using unpaired data, providing a principled basis for paired-sample generation and cross-domain analysis. We validate CASL-VAE on semi-synthetic neuroimaging data, demonstrating improved subtype recovery and paired-sample generation compared to baseline clustering and generative models. We also validate its ability to reveal biologically plausible heterogeneity in Alzheimer's disease.

• The paper introduces a hierarchical latent variable VAE that integrates contrastive analysis to disentangle shared reference and target-specific variations in unpaired data.
• It demonstrates superior unsupervised subtype inference and high-fidelity paired-sample generation, with significant improvements in ARI scores and cosine similarity compared to baseline models.
• Experimental evaluations on semi-synthetic neuroimaging and ADNI datasets confirm that its structured latent space yields clinically meaningful insights while supporting translational applications.

CASL-VAE: Structured Latent Variable Modeling for Unpaired Reference-Target Data

Motivation and Problem Setting

CASL-VAE addresses robust comparative latent modeling between heterogeneous populations from unpaired data, with specific emphasis on domains where quantifying target variation relative to a reference population is vital, but paired samples are unavailable. This paradigm is common in biomedical imaging, genomics, and case-control studies, where comparative analyses are impeded by entangled variability and latent heterogeneity within the target domain (e.g., subtypes of a disease cohort). Standard generative and contrastive analysis (CA) methods typically fail to disentangle these complex multi-modal variations, especially in the absence of explicit supervision, resulting in non-interpretable, unstable latent representations.

Model Architecture and Latent Structure

CASL-VAE integrates contrastive analysis with a hierarchical, structured latent variable architecture within a variational autoencoder framework. The latent space is factorized into:

• Continuous common latent ($z$): Encodes variation shared across reference and target populations, assumed to represent all reference-domain variability.
• Hierarchical salient latent ($S_{cat}, S_{cont}$): Models target-specific heterogeneity via a discrete categorical latent (subtype assignment) and a continuous latent conditioned on subtype (fine-grained intra-subtype variation).

The generative model decodes input as an additive combination of shared (common) and target-specific (salient) reconstructions, forming a clear inductive bias for disentanglement. Salient latents for reference samples are fixed to non-informative preset values, strictly preventing leakage of reference-domain information into target-specific latent variables.

Variational Objective and Regularization

The variational lower bound optimizes approximate joint likelihood over unpaired reference-target observations by leveraging posterior decoupling regularization (Maximum Mean Discrepancy and sample-level consistency). Domain-level binary supervision (reference vs. target) regularizes the discrete salient latent, and a KL divergence penalty on the continuous salient latent for reference samples further enforces target specificity.

The model objective incorporates these constraints, yielding both theoretical justification for the hierarchy and practical support for cross-domain paired generation and subtype analysis.

Inference Modes and Generative Capabilities

CASL-VAE enables three major downstream inference/generation modes:

1. Subtype inference: Unsupervised assignment of target domain samples to discrete subtypes.
2. Conditional paired-sample generation: Generation of domain-paired samples by combining inferred common latent with preset salient states.
3. Synthetic data generation: Simulation of reference-target paired or unpaired synthetic observations by sampling common and salient latents.

This structure allows interpretable cross-domain comparability and generative reasoning in settings with only unpaired marginal observations.

Experimental Evaluation

Semi-synthetic Neuroimaging Data

CASL-VAE was validated on semi-synthetic morphometric MRI data with ground-truth subtype assignments and effect sizes. Compared against standard and deep clustering models (K-means, GMM, GMVAE), semi-supervised deep GANs (Smile-GAN), and generative contrastive VAEs (cVAE, SepVAE), CASL-VAE achieves significantly higher ARI scores for both target-only and joint cluster analysis under mild effect sizes. Notably, in regimes where target effects are subtle and heterogeneity is complex, CASL-VAE's hierarchical latent prior enables superior subtype recovery and separation relative to baseline models.

Furthermore, CASL-VAE demonstrated enhanced paired-sample generation fidelity, achieving highest cosine similarity between generated and ground-truth paired reference samples, reflecting improved semantic consistency and practical generative utility.

Ablation studies confirm robust performance across regularization hyperparameters, with MMD and BCE losses essential for maintaining disentanglement and domain invariance.

Real Alzheimer's Disease Data

Applied to ADNI (Alzheimer's Disease Neuroimaging Initiative) datasets, CASL-VAE recovers clinically plausible heterogeneous atrophy patterns in MCI/AD cohorts relative to healthy controls. Clustering stability analysis supports optimal subtype granularity selection. The salient latent variation index correlates strongly with clinical cognitive scores and AD biomarkers, validating the biological meaningfulness and translational relevance of the inferred latent structure.

Theoretical Implications and Future Directions

CASL-VAE provides a probabilistically principled framework for contrastive generative modeling with hierarchical structured latent variables. This approach elucidates how inductive biases in latent variable decomposition enable simultaneous disentanglement, population heterogeneity modeling, and sample-level pairing—even in the absence of explicit paired training data. The model bridges the gap between contrastive analysis, clustering, and generative reasoning for unpaired-data settings.

Potential future directions include extension to longitudinal data, richer hierarchical latent priors (multi-level ontologies), and adaptation to alternative likelihoods for discrete, categorical, or multi-modal data types. The methodology is generally applicable to reference-target analysis across biomedicine, genomics, and social sciences.

Conclusion

CASL-VAE advances structured generative modeling for comparative analysis with unpaired data. By combining contrastive structural decomposition and hierarchical latent variables, it enables unsupervised subtype discovery, interpretable cross-domain comparison, and high-fidelity paired-sample generation. Empirical results demonstrate strong gains in both clustering and generative tasks, particularly under subtle, heterogeneous effect scenarios. The framework is theoretically grounded and broadly extensible, supporting exploratory population analysis and hypothesis generation in domains where paired supervision is infeasible.

(2607.08254)