ProtoPathway: Biologically Structured Prototype-Pathway Fusion for Multimodal Cancer Survival Prediction

Abstract: We introduce ProtoPathway, an interpretable-by-design multimodal framework for cancer survival prediction that unifies whole slide imaging and transcriptomics through encoders producing biologically grounded representations on both sides of the fusion. On the histopathology side, $K$ learnable morphological prototypes, trained end-to-end with the survival objective, serve as the slide representation itself: patches flow into prototype tokens via soft assignment, compressing variable-length patch sets into fixed task-adaptive tokens. On the genomic side, a bipartite graph neural network encodes gene expression within the Reactome pathway hierarchy, producing pathway embeddings that reflect both constituent genes and their broader biological context through bidirectional message passing over a shared gene--pathway graph. Cross-modal attention then operates over a compact prototype $\times$ pathway matrix in which prototypes query pathways, modeling the biological direction in which molecular programs give rise to tissue morphology. Because both axes carry stable task-learned identity, the attention matrix is itself an interpretability output, yielding native inference-time attribution across the full biological hierarchy, from genes through pathways and prototypes to spatial tissue maps. We evaluate on five TCGA cancer cohorts, demonstrating competitive or superior survival prediction with substantially improved biological interpretability and reduced computational cost, with interpretability claims validated through fold-stratified rank-based population-level analysis. Our source code, model weights, and Reactome pathways, together with a unified codebase reimplementing all multimodal survival baselines under identical preprocessing and evaluation, are available at: https://github.com/AmayaGS/ProtoPathway.

• The paper introduces ProtoPathway, a biologically grounded model that fuses digital pathology with transcriptomics for state-of-the-art cancer survival prediction.
• It employs a novel morphological prototype encoding and a bipartite GNN for pathway embedding, ensuring direct interpretability at every fusion step.
• Experimental results demonstrate improved C-index and efficiency over existing methods, enabling transparent gene-to-tissue inference in clinical settings.

ProtoPathway: Biologically Structured Prototype-Pathway Fusion for Multimodal Cancer Survival Prediction

Introduction and Theoretical Motivation

The ProtoPathway framework addresses a critical challenge in multimodal cancer prognosis: integrating digital pathology and bulk transcriptomics with rigorous, direct interpretability at every step of the predictive pipeline. Existing multimodal survival models have shown that fusing whole slide imaging (WSI) with transcriptomics can yield improved survival prediction in cancer, but have almost universally relied on architectures with either black-box fusion or post-hoc interpretations that lack biological grounding or cross-modal transparency. This paper’s motivating hypothesis is that every intermediate representation in such models should have stable, biologically meaningful semantics, ensuring that interpretability is not bolted on as an afterthought but structurally guaranteed.

Model Architecture

Morphological Prototype Encoding

ProtoPathway introduces an end-to-end trainable morphological prototyping scheme as its WSI encoder. A set of $K$ learnable prototype vectors—initialized via $k$-means and updated by the survival objective—are embedded in feature space and receive input from WSI patch features by soft assignment based on cosine similarity. This mechanism achieves several essential properties:

• Compresses the variable-length patch set into a fixed-size, task-adaptive embedding.
• Prototypes aggregate spatially coherent tissue patterns reflecting prognostically relevant heterogeneity.
• Enables spatially explicit tissue attribution via hard assignment overlays.

Crucially, these prototype tokens serve as the sole morphological representation for downstream reasoning, in contrast to prior work that uses prototypes as auxiliary refiners or selection filters.

Pathway-aware Transcriptomic Encoding

Gene expression encoding is cast as bipartite graph neural network (GNN) message passing over a gene–pathway hierarchy defined by Reactome, supplemented with well-curated Hallmark cancer signatures. The bipartite GNN uses GraphSAGE layers for robust within-pathway aggregation and cross-pathway communication (capitalizing on shared gene membership), followed by a GATv2 attention layer providing per-pathway gene relevance scoring. This implementation introduces two architectural novelties:

• Allows parameter sharing and information propagation across overlapping pathways via shared gene nodes.
• Generates stable, explicit representations for each pathway that reflect both gene expression and pathway topology.

Both the pathway embeddings and pathway importance weights (via gating) are propagated forward, facilitating full-chain genomic attribution.

Interpretable Cross-Modal Fusion

Cross-modal fusion in ProtoPathway is designed to mirror the biological directionality of causation: morphological prototypes query pathway embeddings via multihead attention. This produces a compact, interpretable matrix where each entry quantifies the predicted association between a learned morphological phenotype and a specific molecular pathway. The fusion output is further gated, and final patient-level representation concatenates (i) the fused cross-modal stream, (ii) the isolated genomic embedding, and (iii) the isolated morphological embedding.

Every axis of the fusion matrix—prototypes and pathways—carries task-learned, stable semantic identity, transforming the attention matrix into a direct interpretability output. This enables tracing predictions along a transparent chain from gene to pathway to tissue phenotype to slide location.

Experimental Results

Survival Prediction Performance

Evaluation on five TCGA cohorts (BRCA, BLCA, COADREAD, HNSC, STAD) demonstrates that ProtoPathway delivers the highest overall C-index (0.670), outperforming all contemporary multimodal baselines including MCAT (0.662), MMP (0.659), and SurvPath (0.660), and ranks first on four of five datasets. Notably, ProtoPathway shows clear superiority on HNSC, where all other multimodal methods underperform unimodal gene-only baselines—indicating superior cross-modal integration.

The model achieves this with substantially reduced compute resources: 480K parameters, ~4G FLOPs, 325MB peak VRAM, and 13.6ms/patient during training—resulting in a 28–50$\times$ speedup over the next-fastest cross-modal attention-based baselines. The performance advantage is robust to ablations of the fusion mechanism, with cross-attention fusion outperforming concatenation, bilinear, and gated alternatives both in C-index and in interpretability.

Interpretability Analysis

Morphological Dissection:

ProtoPathway’s unsupervised prototypes in BLCA partition slides into clinically relevant tissue compartments: tumor, tumor stroma, muscle (muscularis propria), connective and adipose tissue, and immune/necrotic regions. Prototype gating analysis reveals that pathway context systematically shifts morphological prioritization, e.g., increasing the importance of microenvironment structures over morphologically dominant tumor compartments in risk stratification—mirroring clinical staging criteria.

Cross-Modal Attribution:

The cross-modal matrix exposes pathway–phenotype associations at inference. For example, in high-risk BLCA slides, prototypes mapping to perivesical fat or muscle are associated with pathways relating to lipid metabolism reprogramming and innate immune suppression (e.g., NFE2L2/NRF2, CLEC7A/NFAT), while low-risk prototypes align with adaptive immune activation and apoptosis engagement.

Gene-level Resolution:

Within-pathway gene attention further drills down to specific effectors: top high-risk pathways highlight mechanistic bottlenecks (PPP3R1, CALM1 in CLEC7A/NFAT; LCK, FYN in FLT3/SRC), aligning with functional studies of tumor immune microenvironment in BLCA.

Spatial Attribution:

Spatial overlays project pathway and gene attributions onto WSI coordinates, providing continuous heatmaps for single genes or pathways. These overlays recapitulate known tumor biology: e.g., strong FGFR2b pathway attention remains tumor-centric, while IL7R attention localizes to immune-rich stroma.

Implications and Future Directions

Practical Implications:

ProtoPathway’s design directly addresses the major translational barrier for AI in digital pathology: the clinical demand for transparent, biologically valid reasoning. Its architecture enables pathologist- and clinician-facing attribution from genetic program to observed tissue, supporting hypothesis generation, biomarker discovery, and risk communication.

Methodological Innovation:

The bipartite GNN and prototype bottleneck advance the field by demonstrating that interpretation can be an architectural property, not just an add-on. The model structure can be immediately extended to other domains requiring hierarchical multi-scale interpretability.

AI Development Trajectories:

Spatially resolved transcriptomics is a natural avenue for future work, enabling fine-grained tissue–gene association beyond the limits of bulk expression. The cross-modal fusion strategy also constitutes a strong template for other high-dimensional, hierarchical multimodal biomedical integration tasks.

Conclusion

ProtoPathway establishes a new standard for interpretable multimodal learning in computational pathology by structuring both the WSI and transcriptomic encoding spaces into stable, task-learned tokens with direct domain semantics. This enables transparent, population-level and patient-level interpretability across the entire gene–pathway–morphology–outcome chain, while delivering state-of-the-art survival prediction with significant computational efficiency gains. The framework's methodological design and empirical results support its utility in both scientific discovery and clinical translation (2605.21454).