---
title: Pathway Activity Autoencoders
url: https://www.emergentmind.com/topics/pathway-activity-autoencoders
type: topic
---

# Pathway Activity Autoencoders

Pathway Activity Autoencoders are autoencoder-based models that compress high-dimensional molecular measurements into pathway-level representations intended to preserve biological semantics while retaining utility for reconstruction, clustering, classification, survival analysis, or other downstream tasks. In the literature, the term covers at least three closely related formulations: generic autoencoders or variational autoencoders whose latent variables are interpreted post hoc as pathway proxies; pathway-structured encoders in which each latent node is explicitly tied to a curated gene set; and unsupervised interpretation pipelines that map nonlinear latent factors to pathways through correlation and enrichment statistics. Across these formulations, the central objective is to replace opaque gene-level embeddings with representations that can be read as pathway activity scores, pathway-associated latent axes, or pathway-weighted sample profiles [1711.04828] [2306.05813] [2601.18604] [2607.05306].

## 1. Conceptual scope and lineage

The motivating problem is the tension between representational capacity and interpretability in omics modeling. Standard AEs and VAEs can learn compact representations of transcriptomic data, but their latent coordinates are usually biologically opaque. Pathway activity autoencoders address this by embedding prior knowledge, or by imposing a pathway interpretation layer after representation learning, so that latent structure can be read in terms of signaling, metabolic, immune, or other curated programs [2306.05813] [2607.05306].

A useful distinction is between **post hoc pathway interpretation** and **pathway-by-design architectures**. In the pan-cancer VAE study built around Tybalt, the model is not pathway-aware by construction: it is a fully connected VAE trained on TCGA RNA-seq, and pathway meaning is assigned later by examining decoder weights and Gene Ontology enrichment of genes associated with selected latent dimensions. In that sense, the latent variables act as unsupervised pathway activity proxies rather than explicitly defined pathway nodes [1711.04828]. By contrast, PAAE and PAVAE impose hard architectural constraints: each pathway has its own encoder, receives only the features mapped to that pathway, and outputs a single scalar activity score. The latent representation is therefore interpretable by construction rather than only after enrichment analysis [2306.05813] [2607.05306].

LaCoGSEA occupies an intermediate position. Its encoder is a standard deterministic autoencoder with no pathway-structured hidden layer, but it defines a dedicated interpretation layer based on global gene–latent Pearson correlations, preranked GSEA, and the aggregation formula \(A = ZW\), where \(Z\) is the sample-by-latent matrix and \(W\) the latent-by-pathway matrix of normalized enrichment scores. The result is an explicit sample-level pathway activity matrix derived from a nonlinear manifold without requiring phenotypic labels [2601.18604].

| Formulation | Pathway relation | Representative paper |
|---|---|---|
| Generic AE/VAE with post hoc enrichment | Latents interpreted after training | [1711.04828] |
| Pathway-structured autoencoder | One scalar per pathway by construction | [2306.05813] |
| Latent-correlation + GSEA framework | Pathways inferred from latent correlations and aggregated into \(A=ZW\) | [2601.18604] |
| Multi-omic pathway-informed autoencoder | Pathway layer used as common integration space across omics | [2607.05306] |

This family resemblance explains why “pathway activity” denotes related but not identical objects across papers. In pathway-structured models it is a learned scalar \(a_p\) produced by a pathway-specific encoder; in LaCoGSEA it is an aggregate of latent activations weighted by pathway enrichments; in generic VAEs it is an inferred biological role attached to a latent feature after decoder-weight analysis and overrepresentation testing.

## 2. Architectural formulations

The canonical pathway-structured formulation is PAAE. Let \(x \in \mathbb{R}^G\) denote the input expression vector and \(P\) a pathway collection. For each pathway \(p\), a pathway-specific encoder \(E_p\) receives only the subvector \(x_p\) corresponding to genes in that pathway and produces a scalar \(a_p\). The pathway activity vector is
\[
a = \big\|_{p \in P} a_p = \big\|_{p \in P} E_p(x_p),
\]
after which the model either treats \(a\) as the main bottleneck or further compresses it into a dense latent \(z\), followed by a decoder that reconstructs the full molecular profile. In the 2023 formulation, the chain is \(x \to a \to z \to \hat{x}\); in the 2026 multi-omics formulation, the architecture is conceptually collapsed to a pathway layer plus decoder when interpretability at the pathway level is primary [2306.05813] [2607.05306].

PAVAE preserves this pathway layer but inserts a variational encoder on top of \(a\). The approximate posterior is factorized as \(q_\phi(z \mid x) = q_\phi(z \mid a)\), so the variational bottleneck is conditional on pathway activities rather than raw genes. This preserves pathway structure while allowing a probabilistic latent representation [2306.05813].

In the original pan-cancer VAE study, the architecture is deliberately generic. All models are fully connected and implemented in Keras with a TensorFlow backend. Three 100-dimensional latent-space VAEs are compared: Tybalt, a one-hidden-layer baseline; Two Hidden VAE (100); and Two Hidden VAE (300). The encoder outputs a mean vector and a standard deviation vector, these are combined by the reparameterization trick to form a sampled \(z\), and the decoder reconstructs the input. No pathway-specific layers, structured priors, or explicit pathway constraints are used [1711.04828].

LaCoGSEA uses a standard deterministic autoencoder rather than a variational or denoising model. Its encoder \(f_\phi: \mathbb{R}^G \to \mathbb{R}^D\) maps an expression profile \(x_i\) to a latent code \(z_i\), and the decoder \(g_\psi\) reconstructs the input. The latent dimension is empirically explored over \(D \in \{1,2,4,8,\dots,128\}\), with \(D=64\) used for most analyses because it lies in a saturation regime where the number of discovered pathways stabilizes [2601.18604].

The multi-omic PAAE extends the pathway layer in two distinct ways. In **early integration**, all omics are concatenated at input,
\[
x = \big\|_{o \in O} x_o,
\]
and each pathway encoder receives all features across all omics that map to genes in the pathway. In **late integration**, one PAAE is trained per omics layer, producing omics-specific pathway activity vectors \(a_o\), which are then combined by either late-mean integration,
\[
\overline{a}_p = \frac{1}{\sum_{o\in O, p\in A_o} 1} \sum_{o\in O, p\in A_o} a_{o,p},
\]
or late-concat integration,
\[
\overline{a} = \big\|_{o\in O} a_o.
\]
This makes pathway space the common coordinate system for multi-omics integration [2607.05306].

## 3. Objectives, regularization, and training dynamics

The simplest objective in this literature is mean squared reconstruction error. For PAAE, the training target is
\[
\min_{\Omega,\Gamma} \left(\hat{x} - x\right)^2,
\]
implemented as MSE over all input features after normalization. The 2023 paper also expresses reconstruction loss as
\[
\mathcal{L}_\text{rec} = \frac{1}{d} \sum_{i=1}^d (x_{:,i} - \hat{x}_{:,i})^2.
\]
Regularization is primarily architectural: one scalar per pathway, a finite-dimensional latent space on top of the pathway layer, dropout, and weight decay. In the 2023 implementation, training used PyTorch 1.12.0 with skorch 0.11.0, Adam with learning rate \(10^{-4}\), 1024 epochs, ReLU after each hidden layer except final layers, and dropout \(0.5\) [2306.05813].

PAVAE and the earlier pan-cancer VAE study use ELBO-based objectives. In the pan-cancer case, “VAE loss is the sum of reconstruction loss and a Kullback-Leibler (KL) divergence term constraining feature activations to a Gaussian distribution,”
\[
\mathcal{L}(x; \theta, \phi)
= \mathbb{E}_{q_\phi(z \mid x)}[\log p_\theta(x \mid z)] - \mathrm{KL}\big(q_\phi(z \mid x) \,\Vert\, p(z)\big).
\]
Training includes warmup through a kappa parameter: “Kappa controls ‘warmup’, which determines how quickly the loss term incorporates KL divergence.” The same paper reports that deeper VAEs have slightly lower validation loss, but the improvement is modest and increasing model depth “likely reduces the ability to interpret the model” [1711.04828].

The PAVAE objective adopts the \(\beta\)-VAE form,
\[
\mathcal{L}(\theta,\phi,x) = \mathbb{E}_{q_{\phi}(z \mid x)}[\log p_{\theta}(x \mid z)] - \beta \, D_\mathrm{KL}\big(q_{\phi}(z \mid x) \Vert p_{\theta}(z)\big),
\]
with both step and smooth KL scheduling. The smooth schedule is defined through a sigmoid \(S_{T_s,T_e}(t)\), with \(T_s = 32\) and \(T_e = 160\). The reported conclusion is not that variational regularization is uniformly helpful; rather, “using vanilla variational frameworks might negatively impact both reconstruction outputs as well as classification performance,” and the dense VAE baseline was substantially worse than the AE baseline unless strong architectural prior knowledge was imposed [2306.05813].

LaCoGSEA adds explicit Elastic Net regularization to the autoencoder objective:
\[
\mathcal{L}(\theta) = \frac{1}{N} \sum_{i=1}^{N} \| x_i - g_\psi(f_\phi(x_i)) \|_2^2 + \lambda_1 \|\theta\|_1 + \lambda_2 \|\theta\|_2^2.
\]
The stated rationale is that \(\lambda_1 \|\theta\|_1\) encourages sparsity and reduces overfitting, while \(\lambda_2 \|\theta\|_2^2\) stabilizes optimization and discourages overly large weights. This is important because the downstream interpretation relies on the geometry of the latent space rather than on a single predictive objective [2601.18604].

The 2026 multi-omic PAAE paper analyzes dropout on the pathway activity layer as a repeatability mechanism. The probability \(Pr(\text{dropout}(a))\) is varied from \(0\) to \(0.9\), and repeatability is quantified using CKA, Pearson and Kendall correlations across runs, and AP@K agreement in pathway rankings. Increasing dropout improves repeatability, with a sharp “phase transition” between 70–80% dropout where absolute Pearson correlation of pathway activities jumps, but excessive dropout reduces ROC AUC and CI and leads to “bleaching” of pathway activity heatmaps. The paper therefore treats pathway-layer dropout as a double-edged regularizer rather than an unambiguous improvement [2607.05306].

## 4. Pathway attribution and activity inference

Pathway activity autoencoders differ most sharply in how they assign pathway meaning to latent structure. In pathway-structured models, the assignment is immediate: \(a_p = E_p(x_p)\) is itself the pathway activity score. Because each encoder sees only its pathway-specific input features and outputs a single scalar, the activity layer \(a = [a_p]_{p\in P}\) is the primary interpretable representation. The 2023 and 2026 PAAE papers emphasize that these scores are learned, continuous summaries of the joint behavior of all genes or features belonging to a pathway, optimized to reconstruct the full profile rather than to reproduce a predefined enrichment statistic. They also note that the sign of \(a_p\) is not a simple activation/inhibition indicator because weights can be positive or negative; interpretation is comparative across samples [2306.05813] [2607.05306].

A second mechanism is post hoc pathway assignment through decoder weights and enrichment analysis. In the pan-cancer VAE study, genes are not assigned to pathways within the encoder or decoder. Instead, the authors use HGSC subtype labels to compute mean latent vectors for mesenchymal and immunoreactive tumors, subtract them to obtain
\[
\Delta = \mu_{\text{Mesenchymal} - \mu_{\text{Immunoreactive}},
\]
identify the most differentiating positive latent feature, map that feature back to genes through decoder weights, select “high weight genes” as those whose weights are \(> 2.5\) standard deviations above the mean for that feature, and run WebGestalt overrepresentation analysis on GO terms. For Tybalt, the top pathway for the most discriminating feature is “Collagen Catabolic Process” with adjusted \(p = 1.8 \times 10^{-9}\); for Two Hidden VAE (300), the top pathway is again “Collagen Catabolic Process” with adjusted \(p = 1.7 \times 10^{-3}\); for Two Hidden VAE (100), the top pathway is “Epidermis Development” with adjusted \(p = 8.0 \times 10^{-4}\). This procedure treats latent arithmetic as a proxy for differences in pathway activity between tumor subgroups [1711.04828].

LaCoGSEA formalizes a third mechanism based on gene–latent correlations. After training the autoencoder, it computes for every gene \(j\) and latent dimension \(k\)
\[
\rho_{j,k} = \mathrm{corr}\big(x_{\cdot,j},\, z_{\cdot,k}\big),
\]
forming a global correlation matrix \(\mathbf{P} \in \mathbb{R}^{G \times D}\). Each latent dimension yields a dense preranked gene list
\[
L_k = \{ g_{(1)}, g_{(2)}, \dots, g_{(G)} \}
\quad \text{s.t.} \quad
\rho_{(1),k} \ge \rho_{(2),k} \ge \dots \ge \rho_{(G),k},
\]
which is fed into classical preranked GSEA. The resulting normalized enrichment scores define a latent–pathway weight matrix \(W \in \mathbb{R}^{D \times M}\), with \(W_{k,p} = \mathrm{NES}_{p,k}\), and sample-level pathway activity is then
\[
A_{i,p} = \sum_{k=1}^{D} z_{i,k} \cdot W_{k,p},
\qquad
A = ZW.
\]
This formulation explicitly allows a pathway to be distributed across multiple latent axes rather than forced into a single unit [2601.18604].

The interpretability literature around PAAE also includes gene-level attribution inside pathway modules. The 2023 paper defines Neural Path Weights
\[
\text{NPW}_j = \prod_{i=1}^k W_{j,i}
\]
and Absolute Neural Path Weights
\[
\text{ANPW}_j[g] = \big| \text{NPW}_j[g] \big|,
\]
which are used to rank genes by their contribution to pathway activity. The paper stresses that nonlinearities make this approximate, but ANPW supports pathway-level biomarker analysis and survival follow-up of top genes [2306.05813].

## 5. Empirical results and application domains

The earliest representative use case is pan-cancer RNA-seq. The pan-cancer VAE paper uses TCGA PanCanAtlas RNA-seq with 10,459 total samples, comprising 9,732 tumors and 727 tumor-adjacent normal samples across 33 cancer types. Inputs for VAE, NMF, and ADAGE are normalized by gene to the \(0\)–\(1\) range; PCA and ICA receive z-score normalization. On a pan-cancer NF1 inactivation task using \(n = 1774\) tumors from BLCA, LGG, LUAD, PCPG, SKCM, and STAD, with elastic net logistic regression and 5-fold cross validation, raw gene expression achieves AUROC \(= 68.4\%\), PCA is the best dimensionality reduction baseline at \(65.6\%\), and the VAE variants, ICA, NMF, and ADAGE show relatively similar performance. The main distinctive signal appears not in classification but in the HGSC subtype analysis, where VAE latent spaces recover extracellular-matrix-associated biology, especially collagen-related processes, in the mesenchymal versus immunoreactive contrast [1711.04828].

The 2023 PAAE/PAVAE study evaluates external generalization from TCGA BRCA to Metabric. TCGA BRCA is used for training with \(n=1217\) samples, Metabric for testing with \(n=1756\), and the shared feature dimension is \(d_x=56{,}867\) genes after mapping and merging. On external Metabric test data, the AE baseline with latent \(z\) and logistic regression yields ROC AUC \(= 0.858\) \((0.023)\) and test MSE \(= 33.758\) \((9.822)\). PAAE (Hallmark) with latent \(z\) and logistic regression yields ROC AUC \(= 0.899\) \((0.011)\) and MSE \(= 25.73\) \((1.13)\); PAAE (KEGG), the best overall model, yields ROC AUC \(= 0.914\) \((0.009)\) and MSE \(= 19.61\) \((1.29)\), with pathway-space performance \(a\) also high at ROC AUC \(= 0.907\) \((0.004)\). The dense VAE baseline is markedly worse, with ROC AUC \(= 0.664\) \((0.049)\) and MSE \(\approx 42{,}009\) \((209{,}076)\). PAVAE performs substantially better than the dense VAE; for KEGG, the pathway activity vector \(a\) with SVM reaches ROC AUC \(= 0.912\) \((0.004)\). The paper also reports that PAAE reaches AE-level performance with fewer parameters and that pathway-structured models significantly outperform AE and VAE baselines in ROC AUC by Wilcoxon testing with \(p < 10^{-3}\) [2306.05813].

LaCoGSEA broadens the application domain from subtype classification to unsupervised pathway enrichment across multiple bulk datasets. It analyzes SCAN-B \((N=7{,}429, G=16{,}736)\), METABRIC \((N=1{,}992, G=16{,}736)\), TCGA Lung NSCLC \((N=1{,}128, G=20{,}531)\), and several GEO cohorts of size \(N=30\)–\(420\). For SCAN-B subtype clustering using KEGG pathway activities and k-means, the reported Adjusted Rand Index versus PAM50 is \(0.372\) for LaCoGSEA, compared with \(0.240\) for PCA-based pathway activity, \(0.126\) for GSVA, and \(0.185\) for ssGSEA. Across five benchmark datasets with independently validated disease pathways, LaCoGSEA achieves mean rank \(\sim 17.5\) and coverage \(\sim 95\%\), whereas standard differential expression has mean rank \(73.6\) and coverage \(29\%\), and AE\_SHAP and AE\_DeepLIFT have mean ranks between \(44.7\) and \(77.0\). In a small liver dataset \((N=30)\), LaCoGSEA ranks Notch signaling and Glycolysis/Gluconeogenesis at rank 1 and uniquely detects MAPK signaling and Adipocytokine signaling [2601.18604].

The 2026 multi-omic PAAE paper places pathway activity in an explicitly translational setting centered on breast cancer survival prediction, subtype classification, and multi-omics integration. It reports that multi-omics PAAEs outperform single-omic PAAEs in median ROC AUC, accuracy, and unsupervised clustering metrics such as Mutual Information and Rand Index; that early integration is best in only 6 of 63 omics-layer combinations for PAM50 classification; that late-mean integration performs best for survival prediction and is the only strategy with consistently positive marginal contributions from all omics layers; and that gene, protein, and microRNA expression layers provide the strongest contributions, whereas mutation data often hurts performance in early integration. For subtype interpretation, the top pathway per subtype is selected by OvR Mutual Information and a decision tree with at most 5 leaf nodes is trained on five pathway activities, yielding a fully interpretable classifier whose rules align with coherent UMAP regions. For survival analysis, pathway activities such as Valine, Leucine, Isoleucine Degradation, Sphingolipid Metabolism, ABC Transporters, Ether Lipid Metabolism, and Glycosaminoglycan Biosynthesis Chondroitin Sulfate are followed by univariate Cox-PH and logrank tests, and genes such as DEGS2, MCCC1, and OXCT1 are highlighted as showing consistent survival associations across TCGA and Metabric [2607.05306].

## 6. Limitations, misconceptions, and open directions

A recurring misconception is that a pathway activity autoencoder necessarily contains one hidden node per pathway. The literature does not support such a single definition. Tybalt-style VAEs are generic, fully connected models whose pathway interpretation is entirely post hoc [1711.04828]. LaCoGSEA also uses an unstructured encoder, but attaches pathway semantics through latent correlations and GSEA [2601.18604]. Only PAAE/PAVAE-style architectures enforce a one-pathway-one-scalar bottleneck by design [2306.05813] [2607.05306].

A second misconception is that pathway activity must be equivalent to an enrichment score such as GSEA, GSVA, or ssGSEA. The pathway-structured PAAE papers explicitly state otherwise: \(a_p\) is a learned, continuous scalar optimized for reconstruction, not a predefined enrichment summary. LaCoGSEA, conversely, does incorporate standard preranked GSEA, but only after learning latent factors; its final activity matrix \(A = ZW\) is therefore a latent–pathway aggregate rather than a direct single-sample enrichment score [2601.18604] [2607.05306].

Several limitations are explicit across the papers. Pathway definitions are incomplete and biased, and gene-to-pathway or omics-to-gene mappings introduce assumptions and noise. The one-scalar-per-pathway design may over-constrain pathways with multimodal or heterogeneous internal structure. The 2023 paper notes that many genes remain outside the pathway union used by the encoder and that pathway hierarchies or pathway–pathway interactions are not encoded explicitly in the architecture [2306.05813]. The 2026 multi-omic study emphasizes non-convex optimization, run-to-run variability, and the fact that dropout improves repeatability only at the cost of possible representational collapse when too large [2607.05306]. The pan-cancer VAE study notes that pathway mapping is post hoc and depends on decoder weights, the \(> 2.5\) standard deviation threshold for high-weight genes, and the enrichment database used; deeper models also likely reduce interpretability [1711.04828]. LaCoGSEA adds that latent axes capture dominant unsupervised variation rather than necessarily the most clinically relevant variation, and that the framework has been validated only on bulk expression rather than single-cell RNA-seq [2601.18604].

The treatment of variational regularization is itself a point of contention. The 2023 study concludes that generic “vanilla” variational frameworks may be harmful for omics reconstruction and classification unless strongly constrained and carefully scheduled, whereas PAVAE shows that pathway-structured variational encoders can partially mitigate this instability [2306.05813]. The earlier pan-cancer work is more measured: deeper VAEs slightly improve validation loss, but the gains are modest and do not translate into clear supervised advantages [1711.04828]. A plausible implication is that pathway activity autoencoders derive most of their benefit from inductive bias and interpretability constraints rather than from variationality alone.

Future directions described in the literature include systematic exploration of latent dimensionality, structured priors or graph-based encoders aligned with pathways, model-agnostic interpretation layers that can be placed on top of VAEs or other nonlinear encoders, multi-omics pathway integration, single-cell and spatial extensions, temporal or perturbational pathway dynamics, and more principled robustness mechanisms than high pathway-layer dropout [1711.04828] [2601.18604] [2607.05306]. The general design principle that emerges is stable across formulations: embed known biological structure where possible, preserve enough nonlinear capacity to capture overlapping transcriptional programs, and define the latent representation so that it remains directly usable for statistical analyses at the pathway level.

Source: https://www.emergentmind.com/topics/pathway-activity-autoencoders