HESCAPE Benchmark for Spatial Transcriptomics

• HESCAPE is a benchmark that standardizes evaluation for cross-modal alignment by pairing histology images with spatial gene expression data.
• It constructs a pan-organ dataset from six targeted gene panels and diverse donors to enable robust multimodal learning.
• Empirical results show that contrastive pretraining boosts mutation classification while degrading gene expression prediction due to batch effects.

Searching arXiv for HESCAPE and related spatial transcriptomics benchmark papers. arXiv search query: "HESCAPE spatial transcriptomics cross-modal contrastive pretraining histology gene expression" HESCAPE most commonly denotes a large-scale benchmark for cross-modal contrastive pretraining in spatial transcriptomics, designed to study joint representation learning from histology images and spatial gene expression profiles. It is built on a curated pan-organ dataset spanning 6 different gene panels and 54 donors, and it standardizes pretraining, retrieval, mutation classification, and gene expression prediction in a single evaluation framework. The benchmark’s central empirical result is deliberately non-uniform: gene expression encoders are the primary determinant of strong representational alignment, and contrastive pretraining consistently improves gene mutation classification performance, yet it degrades direct gene expression prediction compared to baseline encoders trained without cross-modal objectives. The reported analysis identifies batch effects as a key factor that interferes with effective cross-modal alignment, positioning HESCAPE as both a benchmark and a failure-analysis framework for batch-robust multimodal learning in spatial transcriptomics (Gindra et al., 2 Aug 2025).

1. Terminology and scope

In its principal usage, HESCAPE refers to the benchmark introduced in "A Large-Scale Benchmark of Cross-Modal Learning for Histology and Gene Expression in Spatial Transcriptomics" (Gindra et al., 2 Aug 2025). The benchmark focuses on image–transcriptomics contrastive learning, using paired histology patches and spatial gene count vectors to assess how well multimodal encoders align morphology with gene expression and how those representations transfer to downstream tasks.

The term is not globally unique across arXiv. In high-contrast imaging, the ESCAPE project—expanded there as Exoplanetary Systems with a Coronagraphic Archive Processing Engine—is also described as “HESCAPE” when emphasizing its application to the High-contrast Imaging Testbed for Complex Aperture Telescopes (HiCAT) (Lau et al., 2024). A separate, unrelated adaptive-optics usage of ESCAPE denotes Efficient Synthesis of Calibrations for Adaptive Optics through Pseudo-synthetic and Empirical methods (Taylor et al., 2024). In practice, this means that the string “HESCAPE” is ambiguous unless the disciplinary context is specified. Within computational pathology and spatial transcriptomics, however, HESCAPE denotes the benchmark rather than the optical-instrumentation projects.

A common misconception is that the benchmark is primarily a dataset release. The release is substantial, but HESCAPE is more specifically a benchmark: it couples standardized data, evaluation protocols, and benchmarking tools to a controlled comparison of image encoders, gene encoders, contrastive objectives, and downstream transfer settings (Gindra et al., 2 Aug 2025).

2. Data composition and preprocessing

HESCAPE is organized as a pan-organ cohort built from six targeted Xenium panels: 5K, Breast, ImmOnc, Lung, Colon, and Multi-tissue. The data comprise 623,623 unique image–gene pairs, approximately 720 K before deduplication, drawn from 57 samples across 54 donors spanning 8 organs: lung, colon, breast, ovary, lymphoid, pancreas, skin, and heart. Each donor contributes one or more $40\times$ whole-slide FFPE sections, with one fresh-frozen section, imaged at $0.137$–$0.364\,\mu$m/pixel (Gindra et al., 2 Aug 2025).

Panel	Genes
5K	5,001
Breast	280
ImmOnc	380
Lung	343
Colon	322
Multi-tissue	377

The benchmark constructs paired multimodal samples at patch level. Histology images are tiled into $224\times224$ px patches around simulated $55\,\mu$m “pseudo-spots,” and transcripts within each spot are sum-pooled from single-cell Xenium calls to yield a spatial gene count vector. Gene counts then undergo panel-specific normalization or rank-transformation as required by each encoder. The train/validation/test partitioning is patient-wise, ensuring that no donor appears in more than one split (Gindra et al., 2 Aug 2025).

These design choices define the statistical meaning of a HESCAPE sample. The image modality captures local morphology at pseudo-spot scale, while the gene modality represents aggregated spatial counts over the same support. This suggests that HESCAPE is not only a benchmark of encoder quality, but also a benchmark of how well pseudo-spot aggregation preserves biologically useful image–expression correspondence under panel heterogeneity and donor variation.

3. Cross-modal contrastive pretraining framework

HESCAPE evaluates multiple image and gene encoders under a shared contrastive setup. The image encoders all output 128-dim vectors and include CTransPath, UNI, CONCH, Gigapath, and H0mini. Their pretraining regimes span MoCo-style contrastive pretraining on TCGA/PAIP, DINOv2-distilled pretraining on 100 M H&E patches, CLIP-style image–text pretraining on 1.15 M captioned images, DINOv2 on 1.1 B WSI patches, and distillation from H-Optimus-0 on 500 K WSIs. The gene encoders also produce 128-dim outputs and include an MLP trained from scratch on each panel, DRVI as a VAE extension of scVI pre-trained per panel, scFoundation as a frozen transformer pre-trained on bulk and single-cell RNA-seq, and Nicheformer as a frozen BERT-style model pre-trained on single-cell plus targeted spatial data (Gindra et al., 2 Aug 2025).

The contrastive objective is the CLIP InfoNCE loss between image embeddings $v_i$ and gene embeddings $g_i$:

$$L_{\mathrm{CLIP}} = -\frac{1}{2}\left(L_{v\to g} + L_{g\to v}\right),$$

with

$$L_{v\to g} = \frac{1}{|B|}\sum_i -\log \frac{\exp(v_i\cdot g_i/\tau)} {\sum_j \exp(v_i\cdot g_j/\tau)}.$$

Here $\tau$ is a learnable temperature. A SigLip variant is also evaluated, but minimal gains are reported relative to CLIP (Gindra et al., 2 Aug 2025).

The pretraining study includes both frozen and fine-tuned regimes. Full fine-tuning is used for MLP and DRVI, while large transformers are adapted with LoRA-based PEFT. Additional ablations on projection heads—linear, MLP, and small Transformer—together with loss and tuning schedules, indicate that the best alignments occur when image and gene models, especially DRVI, are adapted to each other under CLIP+MLP (Gindra et al., 2 Aug 2025).

The benchmark’s methodological emphasis is therefore not merely on encoder comparison, but on cross-modal adaptation strategy. A plausible implication is that alignment quality in this setting depends less on image-model scale in isolation than on whether the gene encoder supplies a spatially meaningful latent geometry that can be co-adapted to histology features.

4. Downstream evaluation and reported results

HESCAPE evaluates two downstream tasks: gene mutation classification and gene expression prediction. The mutation-classification setup uses a transformer-based slide aggregator over 128-dim patch embeddings with five-fold, site-preserving cross-validation across TCGA cohorts including CRC, BRCA, and LUAD. Performance is reported as F1 score, mean $0.137$0 std. In the Multi-tissue panel comparison between baseline Gigapath and Gigapath-DRVI, the reported changes are CRC-MSI from $0.137$1 to $0.137$2 $0.137$3, CRC-BRAF from $0.137$4 to $0.137$5 $0.137$6, and LUAD-EGFR from $0.137$7 to $0.137$8 $0.137$9. The same section notes that some biomarkers, including CRC-KRAS and BRCA-HER2, underperform, underscoring biomarker-specific effects (Gindra et al., 2 Aug 2025).

The gene-expression-prediction task uses patch features followed by ridge regression to predict the top-50 variable genes under the HEST protocol, with identical train/test splits. Metrics are Pearson’s $0.364\,\mu$0 and MSE averaged across genes and folds. Here the benchmark reports the opposite pattern from mutation classification: cross-modal pretraining often degrades prediction. On the 5K panel, baseline Gigapath reaches $0.364\,\mu$1 $0.364\,\mu$2, whereas DRVI-Gigapath reaches $0.364\,\mu$3 $0.364\,\mu$4 and DRVI-UNI reaches $0.364\,\mu$5 $0.364\,\mu$6. Similar patterns are reported on ImmOnc, Breast, and Lung panels (Gindra et al., 2 Aug 2025).

This asymmetry is one of HESCAPE’s most important contributions. It directly contradicts the common assumption that stronger multimodal alignment should uniformly improve all biologically relevant downstream tasks. In HESCAPE, the same contrastive pretraining that strengthens mutation classification can weaken direct gene expression regression. The benchmark therefore reframes “good multimodal pretraining” as task-conditional rather than monotone.

5. Batch effects and representational failure modes

The benchmark attributes much of this contradiction to batch structure in the gene modality. It reports strong donor- and panel-specific effects in Xenium count matrices and quantifies them using the silhouette-batch score from scIB, where the train/validation/test split is treated as the “batch” label. Across panels, HESCAPE finds a near-linear correlation between silhouette-batch and Recall@5 for Gigapath-DRVI: higher mixing corresponds to better image-to-gene and gene-to-image retrieval, while poor batch integration corresponds to lower retrieval performance (Gindra et al., 2 Aug 2025).

The interpretation given in the benchmark is that uncorrected technical variation in gene data drives spurious alignments, which in turn harm generalizable multimodal representations. This is not a minor implementation detail; it is presented as a central limitation of current cross-modal spatial transcriptomics pipelines. The benchmark’s main cautionary lesson is therefore that retrieval metrics and downstream transfer can diverge when the learned shared space partly encodes batch-correlated nuisance structure rather than biologically stable image–gene correspondence (Gindra et al., 2 Aug 2025).

A second misconception addressed by these findings is that contrastive failure must primarily originate in the image modality. HESCAPE reports instead that gene expression encoders are the primary determinant of strong representational alignment. This shifts methodological attention toward batch-robust gene modeling, integrated latent-variable formulations, and training schedules that do not over-compress morphological information into a batch-contaminated shared space.

6. Toolkit, recommendations, and relation to neighboring uses of the name

HESCAPE is released as an open-source toolkit with standardized Arrow-format datasets on Hugging Face, providing 622 K+ image–gene pairs with metadata, six panels, 54 donors, and train/validation/test splits. The software stack includes preprocessing pipelines built with SpatialData and HuggingFace Datasets for pseudo-spot construction, end-to-end training and evaluation scripts for contrastive pretraining, retrieval metrics including Recall@1, Recall@5, and Recall@10, and downstream pipelines for mutation classification and gene expression prediction. It also supplies benchmarking guidelines for integrating new encoders, loss variants, and batch-corrected gene models (Gindra et al., 2 Aug 2025).

The benchmark’s explicit recommendations are closely aligned with its empirical findings. It recommends explicitly modeling or correcting batch effects in the gene modality, for example through integrated VAEs or adversarial batch-disentanglement, before or during cross-modal pretraining. It also recommends preserving morphological detail in image encoders, since overly aggressive alignment may discard features essential for regression tasks, and it suggests modality-specific fine-tuning schedules to balance alignment against unimodal performance (Gindra et al., 2 Aug 2025).

In a broader terminological sense, HESCAPE should be distinguished from the HiCAT-associated usage in space coronagraphy, where “HESCAPE” refers to the application of the ESCAPE project to the High-contrast Imaging Testbed for Complex Aperture Telescopes, and from the adaptive-optics ESCAPE framework for calibration synthesis on convex adaptive secondary mirrors (Lau et al., 2024, Taylor et al., 2024). Those projects share neither data modality nor objective with the spatial-transcriptomics benchmark. Within biomedical machine learning, HESCAPE specifically names the benchmark that formalizes cross-modal evaluation for histology and gene expression in spatial transcriptomics.

Taken together, HESCAPE functions as both infrastructure and argument. It provides the first large-scale benchmark for image–transcriptomics contrastive learning, but it also shows that benchmark leadership cannot be reduced to alignment scores alone. Its most consequential result is the demonstration that multimodal pretraining can improve one clinically relevant transfer task while degrading another, and that batch effects in the gene modality are a plausible mechanism for that divergence (Gindra et al., 2 Aug 2025).