Pathway Activity Autoencoders

• Pathway Activity Autoencoders are models that compress high-dimensional molecular data into pathway-level representations, enhancing biological interpretability.
• The methodology integrates pathway-structured encoders or post hoc enrichment to map latent features to curated gene sets with clear biological semantics.
• Applications include improved reconstruction, clustering, classification, survival analysis, and multi-omics integration, outperforming generic autoencoders.

Pathway Activity Autoencoders are autoencoder-based models that compress high-dimensional molecular measurements into pathway-level representations intended to preserve biological semantics while retaining utility for reconstruction, clustering, classification, survival analysis, or other downstream tasks. In the literature, the term covers at least three closely related formulations: generic autoencoders or variational autoencoders whose latent variables are interpreted post hoc as pathway proxies; pathway-structured encoders in which each latent node is explicitly tied to a curated gene set; and unsupervised interpretation pipelines that map nonlinear latent factors to pathways through correlation and enrichment statistics. Across these formulations, the central objective is to replace opaque gene-level embeddings with representations that can be read as pathway activity scores, pathway-associated latent axes, or pathway-weighted sample profiles (Way et al., 2017, Avelar et al., 2023, Zheng et al., 26 Jan 2026, Avelar et al., 6 Jul 2026).

1. Conceptual scope and lineage

The motivating problem is the tension between representational capacity and interpretability in omics modeling. Standard AEs and VAEs can learn compact representations of transcriptomic data, but their latent coordinates are usually biologically opaque. Pathway activity autoencoders address this by embedding prior knowledge, or by imposing a pathway interpretation layer after representation learning, so that latent structure can be read in terms of signaling, metabolic, immune, or other curated programs (Avelar et al., 2023, Avelar et al., 6 Jul 2026).

A useful distinction is between post hoc pathway interpretation and pathway-by-design architectures. In the pan-cancer VAE study built around Tybalt, the model is not pathway-aware by construction: it is a fully connected VAE trained on TCGA RNA-seq, and pathway meaning is assigned later by examining decoder weights and Gene Ontology enrichment of genes associated with selected latent dimensions. In that sense, the latent variables act as unsupervised pathway activity proxies rather than explicitly defined pathway nodes (Way et al., 2017). By contrast, PAAE and PAVAE impose hard architectural constraints: each pathway has its own encoder, receives only the features mapped to that pathway, and outputs a single scalar activity score. The latent representation is therefore interpretable by construction rather than only after enrichment analysis (Avelar et al., 2023, Avelar et al., 6 Jul 2026).

LaCoGSEA occupies an intermediate position. Its encoder is a standard deterministic autoencoder with no pathway-structured hidden layer, but it defines a dedicated interpretation layer based on global gene–latent Pearson correlations, preranked GSEA, and the aggregation formula $A = ZW$, where $Z$ is the sample-by-latent matrix and $W$ the latent-by-pathway matrix of normalized enrichment scores. The result is an explicit sample-level pathway activity matrix derived from a nonlinear manifold without requiring phenotypic labels (Zheng et al., 26 Jan 2026).

Formulation	Pathway relation	Representative paper
Generic AE/VAE with post hoc enrichment	Latents interpreted after training	(Way et al., 2017)
Pathway-structured autoencoder	One scalar per pathway by construction	(Avelar et al., 2023)
Latent-correlation + GSEA framework	Pathways inferred from latent correlations and aggregated into $A=ZW$	(Zheng et al., 26 Jan 2026)
Multi-omic pathway-informed autoencoder	Pathway layer used as common integration space across omics	(Avelar et al., 6 Jul 2026)

This family resemblance explains why “pathway activity” denotes related but not identical objects across papers. In pathway-structured models it is a learned scalar $a_p$ produced by a pathway-specific encoder; in LaCoGSEA it is an aggregate of latent activations weighted by pathway enrichments; in generic VAEs it is an inferred biological role attached to a latent feature after decoder-weight analysis and overrepresentation testing.

2. Architectural formulations

The canonical pathway-structured formulation is PAAE. Let $x \in \mathbb{R}^G$ denote the input expression vector and $P$ a pathway collection. For each pathway $p$, a pathway-specific encoder $E_p$ receives only the subvector $x_p$ corresponding to genes in that pathway and produces a scalar $Z$0. The pathway activity vector is

$Z$1

after which the model either treats $Z$2 as the main bottleneck or further compresses it into a dense latent $Z$3, followed by a decoder that reconstructs the full molecular profile. In the 2023 formulation, the chain is $Z$4; in the 2026 multi-omics formulation, the architecture is conceptually collapsed to a pathway layer plus decoder when interpretability at the pathway level is primary (Avelar et al., 2023, Avelar et al., 6 Jul 2026).

PAVAE preserves this pathway layer but inserts a variational encoder on top of $Z$5. The approximate posterior is factorized as $Z$6, so the variational bottleneck is conditional on pathway activities rather than raw genes. This preserves pathway structure while allowing a probabilistic latent representation (Avelar et al., 2023).

In the original pan-cancer VAE study, the architecture is deliberately generic. All models are fully connected and implemented in Keras with a TensorFlow backend. Three 100-dimensional latent-space VAEs are compared: Tybalt, a one-hidden-layer baseline; Two Hidden VAE (100); and Two Hidden VAE (300). The encoder outputs a mean vector and a standard deviation vector, these are combined by the reparameterization trick to form a sampled $Z$7, and the decoder reconstructs the input. No pathway-specific layers, structured priors, or explicit pathway constraints are used (Way et al., 2017).

LaCoGSEA uses a standard deterministic autoencoder rather than a variational or denoising model. Its encoder $Z$8 maps an expression profile $Z$9 to a latent code $W$0, and the decoder $W$1 reconstructs the input. The latent dimension is empirically explored over $W$2, with $W$3 used for most analyses because it lies in a saturation regime where the number of discovered pathways stabilizes (Zheng et al., 26 Jan 2026).

The multi-omic PAAE extends the pathway layer in two distinct ways. In early integration, all omics are concatenated at input,

$W$4

and each pathway encoder receives all features across all omics that map to genes in the pathway. In late integration, one PAAE is trained per omics layer, producing omics-specific pathway activity vectors $W$5, which are then combined by either late-mean integration,

$W$6

or late-concat integration,

$W$7

This makes pathway space the common coordinate system for multi-omics integration (Avelar et al., 6 Jul 2026).

3. Objectives, regularization, and training dynamics

The simplest objective in this literature is mean squared reconstruction error. For PAAE, the training target is

$W$8

implemented as MSE over all input features after normalization. The 2023 paper also expresses reconstruction loss as

$W$9

Regularization is primarily architectural: one scalar per pathway, a finite-dimensional latent space on top of the pathway layer, dropout, and weight decay. In the 2023 implementation, training used PyTorch 1.12.0 with skorch 0.11.0, Adam with learning rate $A=ZW$0, 1024 epochs, ReLU after each hidden layer except final layers, and dropout $A=ZW$1 (Avelar et al., 2023).

PAVAE and the earlier pan-cancer VAE study use ELBO-based objectives. In the pan-cancer case, “VAE loss is the sum of reconstruction loss and a Kullback-Leibler (KL) divergence term constraining feature activations to a Gaussian distribution,”

$A=ZW$2

Training includes warmup through a kappa parameter: “Kappa controls ‘warmup’, which determines how quickly the loss term incorporates KL divergence.” The same paper reports that deeper VAEs have slightly lower validation loss, but the improvement is modest and increasing model depth “likely reduces the ability to interpret the model” (Way et al., 2017).

The PAVAE objective adopts the $A=ZW$3-VAE form,

$A=ZW$4

with both step and smooth KL scheduling. The smooth schedule is defined through a sigmoid $A=ZW$5, with $A=ZW$6 and $A=ZW$7. The reported conclusion is not that variational regularization is uniformly helpful; rather, “using vanilla variational frameworks might negatively impact both reconstruction outputs as well as classification performance,” and the dense VAE baseline was substantially worse than the AE baseline unless strong architectural prior knowledge was imposed (Avelar et al., 2023).

LaCoGSEA adds explicit Elastic Net regularization to the autoencoder objective: $A=ZW$8 The stated rationale is that $A=ZW$9 encourages sparsity and reduces overfitting, while $a_p$0 stabilizes optimization and discourages overly large weights. This is important because the downstream interpretation relies on the geometry of the latent space rather than on a single predictive objective (Zheng et al., 26 Jan 2026).

The 2026 multi-omic PAAE paper analyzes dropout on the pathway activity layer as a repeatability mechanism. The probability $a_p$1 is varied from $a_p$2 to $a_p$3, and repeatability is quantified using CKA, Pearson and Kendall correlations across runs, and AP@K agreement in pathway rankings. Increasing dropout improves repeatability, with a sharp “phase transition” between 70–80% dropout where absolute Pearson correlation of pathway activities jumps, but excessive dropout reduces ROC AUC and CI and leads to “bleaching” of pathway activity heatmaps. The paper therefore treats pathway-layer dropout as a double-edged regularizer rather than an unambiguous improvement (Avelar et al., 6 Jul 2026).

4. Pathway attribution and activity inference

Pathway activity autoencoders differ most sharply in how they assign pathway meaning to latent structure. In pathway-structured models, the assignment is immediate: $a_p$4 is itself the pathway activity score. Because each encoder sees only its pathway-specific input features and outputs a single scalar, the activity layer $a_p$5 is the primary interpretable representation. The 2023 and 2026 PAAE papers emphasize that these scores are learned, continuous summaries of the joint behavior of all genes or features belonging to a pathway, optimized to reconstruct the full profile rather than to reproduce a predefined enrichment statistic. They also note that the sign of $a_p$6 is not a simple activation/inhibition indicator because weights can be positive or negative; interpretation is comparative across samples (Avelar et al., 2023, Avelar et al., 6 Jul 2026).

A second mechanism is post hoc pathway assignment through decoder weights and enrichment analysis. In the pan-cancer VAE study, genes are not assigned to pathways within the encoder or decoder. Instead, the authors use HGSC subtype labels to compute mean latent vectors for mesenchymal and immunoreactive tumors, subtract them to obtain

$a_p$7

identify the most differentiating positive latent feature, map that feature back to genes through decoder weights, select “high weight genes” as those whose weights are $a_p$8 standard deviations above the mean for that feature, and run WebGestalt overrepresentation analysis on GO terms. For Tybalt, the top pathway for the most discriminating feature is “Collagen Catabolic Process” with adjusted $a_p$9; for Two Hidden VAE (300), the top pathway is again “Collagen Catabolic Process” with adjusted $x \in \mathbb{R}^G$0; for Two Hidden VAE (100), the top pathway is “Epidermis Development” with adjusted $x \in \mathbb{R}^G$1. This procedure treats latent arithmetic as a proxy for differences in pathway activity between tumor subgroups (Way et al., 2017).

LaCoGSEA formalizes a third mechanism based on gene–latent correlations. After training the autoencoder, it computes for every gene $x \in \mathbb{R}^G$2 and latent dimension $x \in \mathbb{R}^G$3

$x \in \mathbb{R}^G$4

forming a global correlation matrix $x \in \mathbb{R}^G$5. Each latent dimension yields a dense preranked gene list

$x \in \mathbb{R}^G$6

which is fed into classical preranked GSEA. The resulting normalized enrichment scores define a latent–pathway weight matrix $x \in \mathbb{R}^G$7, with $x \in \mathbb{R}^G$8, and sample-level pathway activity is then

$x \in \mathbb{R}^G$9

This formulation explicitly allows a pathway to be distributed across multiple latent axes rather than forced into a single unit (Zheng et al., 26 Jan 2026).

The interpretability literature around PAAE also includes gene-level attribution inside pathway modules. The 2023 paper defines Neural Path Weights

$P$0

and Absolute Neural Path Weights

$P$1

which are used to rank genes by their contribution to pathway activity. The paper stresses that nonlinearities make this approximate, but ANPW supports pathway-level biomarker analysis and survival follow-up of top genes (Avelar et al., 2023).

5. Empirical results and application domains

The earliest representative use case is pan-cancer RNA-seq. The pan-cancer VAE paper uses TCGA PanCanAtlas RNA-seq with 10,459 total samples, comprising 9,732 tumors and 727 tumor-adjacent normal samples across 33 cancer types. Inputs for VAE, NMF, and ADAGE are normalized by gene to the $P$2–$P$3 range; PCA and ICA receive z-score normalization. On a pan-cancer NF1 inactivation task using $P$4 tumors from BLCA, LGG, LUAD, PCPG, SKCM, and STAD, with elastic net logistic regression and 5-fold cross validation, raw gene expression achieves AUROC $P$5, PCA is the best dimensionality reduction baseline at $P$6, and the VAE variants, ICA, NMF, and ADAGE show relatively similar performance. The main distinctive signal appears not in classification but in the HGSC subtype analysis, where VAE latent spaces recover extracellular-matrix-associated biology, especially collagen-related processes, in the mesenchymal versus immunoreactive contrast (Way et al., 2017).

The 2023 PAAE/PAVAE study evaluates external generalization from TCGA BRCA to Metabric. TCGA BRCA is used for training with $P$7 samples, Metabric for testing with $P$8, and the shared feature dimension is $P$9 genes after mapping and merging. On external Metabric test data, the AE baseline with latent $p$0 and logistic regression yields ROC AUC $p$1 $p$2 and test MSE $p$3 $p$4. PAAE (Hallmark) with latent $p$5 and logistic regression yields ROC AUC $p$6 $p$7 and MSE $p$8 $p$9; PAAE (KEGG), the best overall model, yields ROC AUC $E_p$0 $E_p$1 and MSE $E_p$2 $E_p$3, with pathway-space performance $E_p$4 also high at ROC AUC $E_p$5 $E_p$6. The dense VAE baseline is markedly worse, with ROC AUC $E_p$7 $E_p$8 and MSE $E_p$9 $x_p$0. PAVAE performs substantially better than the dense VAE; for KEGG, the pathway activity vector $x_p$1 with SVM reaches ROC AUC $x_p$2 $x_p$3. The paper also reports that PAAE reaches AE-level performance with fewer parameters and that pathway-structured models significantly outperform AE and VAE baselines in ROC AUC by Wilcoxon testing with $x_p$4 (Avelar et al., 2023).

LaCoGSEA broadens the application domain from subtype classification to unsupervised pathway enrichment across multiple bulk datasets. It analyzes SCAN-B $x_p$5, METABRIC $x_p$6, TCGA Lung NSCLC $x_p$7, and several GEO cohorts of size $x_p$8–$x_p$9. For SCAN-B subtype clustering using KEGG pathway activities and k-means, the reported Adjusted Rand Index versus PAM50 is $Z$00 for LaCoGSEA, compared with $Z$01 for PCA-based pathway activity, $Z$02 for GSVA, and $Z$03 for ssGSEA. Across five benchmark datasets with independently validated disease pathways, LaCoGSEA achieves mean rank $Z$04 and coverage $Z$05, whereas standard differential expression has mean rank $Z$06 and coverage $Z$07, and AE_SHAP and AE_DeepLIFT have mean ranks between $Z$08 and $Z$09. In a small liver dataset $Z$10, LaCoGSEA ranks Notch signaling and Glycolysis/Gluconeogenesis at rank 1 and uniquely detects MAPK signaling and Adipocytokine signaling (Zheng et al., 26 Jan 2026).

The 2026 multi-omic PAAE paper places pathway activity in an explicitly translational setting centered on breast cancer survival prediction, subtype classification, and multi-omics integration. It reports that multi-omics PAAEs outperform single-omic PAAEs in median ROC AUC, accuracy, and unsupervised clustering metrics such as Mutual Information and Rand Index; that early integration is best in only 6 of 63 omics-layer combinations for PAM50 classification; that late-mean integration performs best for survival prediction and is the only strategy with consistently positive marginal contributions from all omics layers; and that gene, protein, and microRNA expression layers provide the strongest contributions, whereas mutation data often hurts performance in early integration. For subtype interpretation, the top pathway per subtype is selected by OvR Mutual Information and a decision tree with at most 5 leaf nodes is trained on five pathway activities, yielding a fully interpretable classifier whose rules align with coherent UMAP regions. For survival analysis, pathway activities such as Valine, Leucine, Isoleucine Degradation, Sphingolipid Metabolism, ABC Transporters, Ether Lipid Metabolism, and Glycosaminoglycan Biosynthesis Chondroitin Sulfate are followed by univariate Cox-PH and logrank tests, and genes such as DEGS2, MCCC1, and OXCT1 are highlighted as showing consistent survival associations across TCGA and Metabric (Avelar et al., 6 Jul 2026).

6. Limitations, misconceptions, and open directions

A recurring misconception is that a pathway activity autoencoder necessarily contains one hidden node per pathway. The literature does not support such a single definition. Tybalt-style VAEs are generic, fully connected models whose pathway interpretation is entirely post hoc (Way et al., 2017). LaCoGSEA also uses an unstructured encoder, but attaches pathway semantics through latent correlations and GSEA (Zheng et al., 26 Jan 2026). Only PAAE/PAVAE-style architectures enforce a one-pathway-one-scalar bottleneck by design (Avelar et al., 2023, Avelar et al., 6 Jul 2026).

A second misconception is that pathway activity must be equivalent to an enrichment score such as GSEA, GSVA, or ssGSEA. The pathway-structured PAAE papers explicitly state otherwise: $Z$11 is a learned, continuous scalar optimized for reconstruction, not a predefined enrichment summary. LaCoGSEA, conversely, does incorporate standard preranked GSEA, but only after learning latent factors; its final activity matrix $Z$12 is therefore a latent–pathway aggregate rather than a direct single-sample enrichment score (Zheng et al., 26 Jan 2026, Avelar et al., 6 Jul 2026).

Several limitations are explicit across the papers. Pathway definitions are incomplete and biased, and gene-to-pathway or omics-to-gene mappings introduce assumptions and noise. The one-scalar-per-pathway design may over-constrain pathways with multimodal or heterogeneous internal structure. The 2023 paper notes that many genes remain outside the pathway union used by the encoder and that pathway hierarchies or pathway–pathway interactions are not encoded explicitly in the architecture (Avelar et al., 2023). The 2026 multi-omic study emphasizes non-convex optimization, run-to-run variability, and the fact that dropout improves repeatability only at the cost of possible representational collapse when too large (Avelar et al., 6 Jul 2026). The pan-cancer VAE study notes that pathway mapping is post hoc and depends on decoder weights, the $Z$13 standard deviation threshold for high-weight genes, and the enrichment database used; deeper models also likely reduce interpretability (Way et al., 2017). LaCoGSEA adds that latent axes capture dominant unsupervised variation rather than necessarily the most clinically relevant variation, and that the framework has been validated only on bulk expression rather than single-cell RNA-seq (Zheng et al., 26 Jan 2026).

The treatment of variational regularization is itself a point of contention. The 2023 study concludes that generic “vanilla” variational frameworks may be harmful for omics reconstruction and classification unless strongly constrained and carefully scheduled, whereas PAVAE shows that pathway-structured variational encoders can partially mitigate this instability (Avelar et al., 2023). The earlier pan-cancer work is more measured: deeper VAEs slightly improve validation loss, but the gains are modest and do not translate into clear supervised advantages (Way et al., 2017). A plausible implication is that pathway activity autoencoders derive most of their benefit from inductive bias and interpretability constraints rather than from variationality alone.

Future directions described in the literature include systematic exploration of latent dimensionality, structured priors or graph-based encoders aligned with pathways, model-agnostic interpretation layers that can be placed on top of VAEs or other nonlinear encoders, multi-omics pathway integration, single-cell and spatial extensions, temporal or perturbational pathway dynamics, and more principled robustness mechanisms than high pathway-layer dropout (Way et al., 2017, Zheng et al., 26 Jan 2026, Avelar et al., 6 Jul 2026). The general design principle that emerges is stable across formulations: embed known biological structure where possible, preserve enough nonlinear capacity to capture overlapping transcriptional programs, and define the latent representation so that it remains directly usable for statistical analyses at the pathway level.