hist2RNA: Deep Learning for Transcriptome Prediction

• hist2RNA is a deep learning framework that infers bulk gene expression from histopathology images through end-to-end mapping of image features to transcriptomic profiles.
• It employs convolutional backbones and generative models, achieving high correlation metrics in breast cancer and pancancer studies with streamlined computational protocols.
• By integrating pathway-level interpretability and robust evaluation metrics, hist2RNA supports non-destructive molecular profiling for enhanced clinical diagnostics and personalized therapy.

hist2RNA refers to a class of deep learning architectures designed for transcriptome prediction—specifically, the inference of bulk RNA-sequencing (RNA-seq) gene expression—from digitized histopathology whole-slide images (WSIs). The approach is motivated by the need to non-destructively and cost-efficiently estimate the molecular states of tissues, such as in cancer, using computational analysis of routinely acquired histological slides. Methods under the hist2RNA umbrella perform direct end-to-end mapping from image-derived features to genome-scale (or panel-scale) gene expression profiles and, in recent variants, enable uncertainty quantification and biological interpretability. Notable implementations include the original hist2RNA regression pipeline targeting breast cancer gene sets (Mondol et al., 2023) and flow-matching–based generative models for genome-wide prediction (Song et al., 12 May 2026).

1. Data Modalities, Preprocessing, and Feature Extraction

Hist2RNA workflows process paired datasets of WSIs and matched bulk RNA-seq measurements. WSIs are high-resolution images (typically hematoxylin and eosin stained, 0.25 µm/pixel, ~×40 magnification), which are downsampled for efficiency. Tumor regions are manually delineated to exclude necrotic tissue and ensure accurate spatial correspondence. Non-overlapping patches (e.g., 224×224 px or 256×256 px) are sampled, often numbering in the thousands per slide. Colour normalization techniques (Macenko, Vahadane) are applied to correct for batch effects in staining.

For transcriptomic targets, raw RNA-seq counts are log-transformed, retaining only those genes relevant to the predictive task (e.g., 138 genes from breast cancer commercial panels (Mondol et al., 2023), or genome-wide profiles (Song et al., 12 May 2026)).

Patch-level visual features are derived from convolutional backbones (EfficientNet-b0, ResNet-50, or unified multimodal encoders). For each patch $x_{p,i}\in\mathbb{R}^F$, features are aggregated to slide-level descriptors by mean pooling:

$z_p = \frac{1}{N_p}\sum_{i=1}^{N_p} x_{p,i}$

or via more general weighted pooling. For large-scale settings, molecular clustering of tile features (e.g., k-means with $K=100$) yields a compressed WSI representation $Y\in\mathbb{R}^{100\times d}$.

2. Predictive Architectures and Learning Objectives

The canonical hist2RNA model for breast cancer prediction utilizes a lightweight convolutional regressor accepting the aggregated image feature vector $z_p$, passing it through three 1D convolutional blocks and a linear layer to output a gene expression vector $\hat{y}_p\in\mathbb{R}^{138}$ (Mondol et al., 2023). The training objective is mean squared error (MSE) loss across patients and genes:

$$\mathcal{L}(\theta) = \frac{1}{P}\sum_{p=1}^P \frac{1}{G}\sum_{g=1}^G \left(\hat{y}_{g,p} - y_{g,p}\right)^2$$

Advanced generative approaches, such as RNA-FM (Song et al., 12 May 2026), move beyond deterministic regression and model the full conditional distribution $p(y\mid I)$ via continuous-time flow-matching. A velocity field network $v_\theta$ parameterizes the probability-flow ordinary differential equation (ODE):

$$\frac{dX_t}{dt} = v_\theta(X_t, t \mid I), \quad X_{t=0}\sim p_0, \quad X_{t=1}\sim p_{\mathrm{gene}}(\cdot \mid I)$$

The training loss aligns learned velocities with the (analytically known) target velocities under a given interpolant (usually linear):

$z_p = \frac{1}{N_p}\sum_{i=1}^{N_p} x_{p,i}$0

The RNA-FM architecture deploys conditional transformers (DiT blocks) to model gene–gene and gene–pathway dependencies, with histological context encoded through cross-attention or FiLM layers.

3. Biological Structure and Interpretability

Pathway-level biological structure is explicitly modeled in advanced hist2RNA architectures. Genes are grouped into pathway sets $z_p = \frac{1}{N_p}\sum_{i=1}^{N_p} x_{p,i}$1 according to Gene Ontology (Biological Process), and pathway tokens are constructed:

$z_p = \frac{1}{N_p}\sum_{i=1}^{N_p} x_{p,i}$2

where $z_p = \frac{1}{N_p}\sum_{i=1}^{N_p} x_{p,i}$3 comprises gene expression values for pathway $z_p = \frac{1}{N_p}\sum_{i=1}^{N_p} x_{p,i}$4, $z_p = \frac{1}{N_p}\sum_{i=1}^{N_p} x_{p,i}$5 is a small MLP, and $z_p = \frac{1}{N_p}\sum_{i=1}^{N_p} x_{p,i}$6 is a learned embedding. An inter-pathway attention graph restricts message-passing to related pathways (adjacency from gene overlap), preserving interpretability and enforcing biologically plausible communication. Pathway-to-gene reconstruction decodes pathway tokens back to gene-level predictions, averaging out overlapping genes.

This pathway-centric tokenization is critical to making gene-wide models tractable and enables visualizations such as chord diagrams and enrichment plots that elucidate the morphology–transcriptome interface (Song et al., 12 May 2026).

4. Training Protocols, Inference, and Computational Efficiency

Training follows standard supervised or conditional generative protocols. Regression models are optimized with Adam or AdamW, with common learning rates ($z_p = \frac{1}{N_p}\sum_{i=1}^{N_p} x_{p,i}$7 for regression, $z_p = \frac{1}{N_p}\sum_{i=1}^{N_p} x_{p,i}$8 for generative), batch sizes from 12–32, and early stopping or weight EMA applied.

Inference on new WSIs requires only feature extraction from ∼1,000 patches (for regression) or up to ∼4,000 (for clustering-based encoders), followed by a single forward pass (regression) or ODE integration (generative). RNA-FM employs backward Euler integration in 20 steps, plus classifier-free guidance for controllable sampling.

Notably, hist2RNA provides substantial computational advantages over separate gene-specific pipelines: training completes in under an hour on a single GPU, and inference on new samples is typically <60 s, with parameter counts (∼10 M) and energy usage (∼0.2 kWh) significantly reduced compared to patch-based ResNet ensembles (Mondol et al., 2023).

5. Evaluation Metrics, Baselines, and Empirical Results

Model performance is evaluated on internal TCGA (training/test split) and external tissue microarray (TMA) datasets. Both Pearson/Spearman correlation coefficients are reported—“across patients” (overall agreement for each patient’s multigene profile) and “across genes” (per-gene accuracy across patients). Additional metrics include RMSE, $z_p = \frac{1}{N_p}\sum_{i=1}^{N_p} x_{p,i}$9, interval coverage for uncertainty calibration, Gaussian NLL, and error–variance rank correlation.

Key empirical results include:

• Median Spearman $K=100$0 across patients, $K=100$1 across genes for breast cancer gene panels (EfficientNet backbone) (Mondol et al., 2023).
• On RNA-FM, PCC = 0.92 on top-100 genes (cross-validation) and 0.50 on top-50 genes (external CPTAC) (Song et al., 12 May 2026).
• Ablation results demonstrate critical dependence on pathway patchification and inter-pathway graph—removing these reduces PCC by 0.05 and 0.02, respectively.
• For survival and clinical tasks: luminal subtype prediction achieves c-index = 0.56 (univariate), 0.65 (multivariate), hazard ratio ∼2.0 (Luminal B vs A), with predicted biomarkers (ESR1, PGR, ERBB2, MKI67) correlating with immunohistochemistry status and tumor grade.
• Calibration: 50/80/90 % prediction intervals match nominal coverage, with error–variance Spearman correlation ≈ +0.6.

RNA-FM and related methods consistently outperform regression models and previously published baselines such as HE2RNA, tRNAsformer, and SEQUOIA, both in accuracy and biological concordance (Song et al., 12 May 2026).

6. Clinical and Biological Implications

Hist2RNA methodologies bridge morphologic and molecular diagnostics, facilitating rapid estimation of clinically actionable gene expression profiles from standard pathology workflows—circumventing costs, delays, and tissue consumption associated with RNA-seq. Models generalize to external cohorts, demonstrate robust domain transfer (e.g., from TCGA to CPTAC or TMA collections), and maintain prognostic and biological signal fidelity. This enables downstream phenotype classification (e.g., PAM50 subtypes), risk stratification via Cox proportional hazards modeling, and biomarker verification against ground-truth immunohistochemistry.

A plausible implication is the potential integration of deep morpho-transcriptomic prediction into clinical pipelines, reducing sequencing dependency in ER+ breast cancer care and possibly other tumoral settings.

7. Limitations and Future Directions

The original hist2RNA model is limited in gene coverage (restricted to 138 genes) and uses a purely deterministic regression mapping, which constrains its capacity to express biological variability and model uncertainty (Mondol et al., 2023). Next-generation methods such as RNA-FM address these limitations through probabilistic, generative modeling and explicit pathway integration (Song et al., 12 May 2026). Future directions include scaling to single-cell and spatial transcriptomics targets, further interpretability enhancements, and real-world translational validation across diverse cancer and organ systems.

Model	Domain	Gene Coverage	Key Metric (PCC)	Uncertainty Modeling
hist2RNA	Breast cancer	138	0.82 (patients)	No
RNA-FM	Pancancer	20,000	0.92 (top 100)	Yes

This suggests that recent advances markedly improve both model scale and clinical relevance by leveraging generative frameworks and pathway-level interpretability.