SPATIA: Multimodal Spatial Transcriptomics Model

• SPATIA is a multi-scale model that unifies cell images, gene expression data, and spatial coordinates to reveal biological architecture from single cells to tissues.
• The model employs hierarchical attention fusion across cell, niche, and tissue levels to enhance tasks such as cell annotation, clustering, and gene expression prediction.
• Its generative module synthesizes high-resolution cell images conditioned on gene expression and speeds up inference by merging less important tokens.

SPATIA is a multi-scale, multimodal model for image-based spatial transcriptomics that learns unified, spatially aware representations of single cells, local neighborhoods, and whole tissues by jointly modeling cell-resolved morphology, gene expression, and spatial context. In the reported formulation, it supports both predictive tasks—such as cell annotation, clustering, gene expression prediction, cross-modal prediction, and biomarker status prediction—and generative tasks that synthesize high-resolution cell images conditioned on gene expression. The model is trained on a multi-scale dataset consisting of 17 million cell-gene pairs, 1 million niche-gene pairs, and 10,000 tissue-gene pairs across 49 donors, 17 tissue types, and 12 disease states (Kong et al., 7 Jul 2025).

1. Biological setting and multi-scale data model

SPATIA is designed for settings in which cellular phenotype is governed jointly by intrinsic morphology, molecular state, and spatial organization. The underlying data are image-based spatial transcriptomics measurements, with each cell represented by a cropped cell image, a gene expression vector, and a spatial coordinate. Formally, the cell-level dataset is

$$\mathcal{D}_{\text{cell}} = \{(C_i, g_i, s_i)\}_{i=1}^M,$$

where $C_i \in \mathbb{R}^{H \times W \times 3}$ is a high-resolution RGB cropped image, $g_i \in \mathbb{R}^G$ is a gene expression vector, and $s_i = (x_i, y_i) \in \mathbb{R}^2$ is a 2D spatial coordinate (Kong et al., 7 Jul 2025).

The MIST dataset organizes these measurements at three hierarchical scales. At the cell level, MIST-C contains approximately 17 million cell-gene pairs. At the niche level, MIST-N contains approximately 1 million niche-gene pairs obtained by tiling slides into non-overlapping $256 \times 256$ pixel niches and aggregating expression within each tile. At the tissue level, MIST-T contains 10,000 tissue-gene entries summarizing whole-slide or large-region organization (Kong et al., 7 Jul 2025).

Scale	Representation	Reported size
Cell	Cell image, cell gene vector, cell coordinate	~17 million cell-gene pairs
Niche	$256 \times 256$ image patch, pooled gene profile, niche coordinates	~1 million niche-gene pairs
Tissue	Collection of niche embeddings with positional metadata and aggregated expression	10,000 tissue-gene pairs

A niche $j$ is defined by a fixed-size square region, an image patch $N_j$, a cell set $\mathcal{C}_j$, and an aggregated gene expression vector

$g^n_j = \sum_{i \in \mathcal{C}_j} g_i.$

This construction makes local microenvironmental context an explicit modeling object rather than an implicit by-product of cell-level features. At the tissue level, SPATIA uses collections of niche representations and their positions within the slide to capture large-scale architecture (Kong et al., 7 Jul 2025).

2. Hierarchical architecture and multimodal fusion

SPATIA combines modality-specific encoders, cross-attention fusion, and hierarchical spatial transformers. At the cell level, cell images are processed by a ViT-based image encoder $C_i \in \mathbb{R}^{H \times W \times 3}$0, yielding a sequence of morphological tokens

$C_i \in \mathbb{R}^{H \times W \times 3}$1

The reported implementation uses ViTMAEForPreTraining, with encoder hidden size 768, 12 layers, MLP intermediate size 3072, decoder hidden size 512, and 8 decoder layers (Kong et al., 7 Jul 2025).

Gene expression is encoded by a gene encoder $C_i \in \mathbb{R}^{H \times W \times 3}$2 based on scPRINT: $C_i \in \mathbb{R}^{H \times W \times 3}$3 The reported gene encoder uses embedding dimension 256, vocabulary size 23,122 genes, dropout 0.1, and a FlashTransformer with 8 blocks, hidden size 256, and intermediate size 1024 (Kong et al., 7 Jul 2025).

Cell-level multimodal fusion is implemented with cross-attention, using image tokens as queries and the gene embedding as keys and values: $C_i \in \mathbb{R}^{H \times W \times 3}$4 This yields a unified single-cell embedding that aligns fine-grained morphology with molecular state. The niche level then aggregates cell embeddings inside each niche,

$C_i \in \mathbb{R}^{H \times W \times 3}$5

and fuses them with niche image tokens: $C_i \in \mathbb{R}^{H \times W \times 3}$6 At the tissue level, SPATIA processes niche embeddings with added 2D positional encodings and produces tissue-contextualized representations through another cross-attention stage,

$C_i \in \mathbb{R}^{H \times W \times 3}$7

The final unified cell embedding combines cell, niche, and tissue information: $C_i \in \mathbb{R}^{H \times W \times 3}$8 The reported fusion layers use image fusion dimension 768, expression fusion dimension 256, and dropout 0.1 (Kong et al., 7 Jul 2025).

This hierarchy makes spatial context explicit at multiple biological scales. A plausible implication is that SPATIA treats spatial organization not merely as coordinates appended to a feature vector, but as a structured dependency spanning single-cell morphology, local microenvironment, and tissue architecture.

3. Generative decoder, token merging, and optimization

SPATIA includes a conditional diffusion model for gene-conditioned cell image generation. Given a cell-gene pair $C_i \in \mathbb{R}^{H \times W \times 3}$9 and a modified gene vector $g_i \in \mathbb{R}^G$0, the model defines a conditioning embedding

$g_i \in \mathbb{R}^G$1

A frozen VAE maps images into latent space, and a lightweight U-Net of approximately 40M parameters denoises those latents under cross-attention conditioning on $g_i \in \mathbb{R}^G$2 (Kong et al., 7 Jul 2025).

The diffusion objective is the standard latent denoising loss: $g_i \in \mathbb{R}^G$3 where $g_i \in \mathbb{R}^G$4. To steer generation toward the condition while preserving diversity, SPATIA uses classifier-free guidance: $g_i \in \mathbb{R}^G$5 Token importance is then defined through the absolute difference between conditional and unconditional noise predictions: $g_i \in \mathbb{R}^G$6 These scores drive a training-free token-merging procedure during inference (Kong et al., 7 Jul 2025).

The token-merging mechanism preserves the top $g_i \in \mathbb{R}^G$7 most important tokens as anchors and merges less important tokens according to cosine similarity in query-key space,

$g_i \in \mathbb{R}^G$8

followed by gated similarity,

$g_i \in \mathbb{R}^G$9

Unlike exclusive many-to-one merging, the reported method allows overlaps, so a token can be associated with multiple anchor tokens. The reported result is approximately 37% faster generation without loss of image quality, as measured by PSNR and SSIM (Kong et al., 7 Jul 2025).

Pretraining combines self-supervised reconstruction with diffusion. Image reconstruction uses

$s_i = (x_i, y_i) \in \mathbb{R}^2$0

and the total reconstruction objective is

$s_i = (x_i, y_i) \in \mathbb{R}^2$1

The hierarchical modules are trained concurrently, with AdamW at learning rate $s_i = (x_i, y_i) \in \mathbb{R}^2$2 for the main model; the diffusion U-Net uses AdamW with learning rate $s_i = (x_i, y_i) \in \mathbb{R}^2$3, cosine decay, 500 warm-up steps, batch size 256, and 25k steps on 3M cell-gene pairs sampled from MIST-C (Kong et al., 7 Jul 2025).

4. Benchmarks, reported performance, and ablations

SPATIA is benchmarked against 13 existing models across 12 individual tasks spanning cell annotation, cell clustering, gene imputation, cross-modal prediction, and image generation. On cell annotation in the Multiple Sclerosis dataset, SPATIA reports F1 $s_i = (x_i, y_i) \in \mathbb{R}^2$4 and precision $s_i = (x_i, y_i) \in \mathbb{R}^2$5, compared with scGPT at $s_i = (x_i, y_i) \in \mathbb{R}^2$6, CellPLM at $s_i = (x_i, y_i) \in \mathbb{R}^2$7, scBERT at $s_i = (x_i, y_i) \in \mathbb{R}^2$8, and CellTypist at $s_i = (x_i, y_i) \in \mathbb{R}^2$9. On cell clustering, it reports ARI $256 \times 256$0 and NMI $256 \times 256$1, compared with CellPLM at $256 \times 256$2, scGPT at $256 \times 256$3, PCA at $256 \times 256$4, and Geneformer at $256 \times 256$5 (Kong et al., 7 Jul 2025).

In biomarker status prediction on the BCNB invasive breast cancer cohort, SPATIA reports the highest AUC and balanced accuracy for ER, PR, and HER2. The reported values are $256 \times 256$6 for ER, $256 \times 256$7 for PR, and $256 \times 256$8 for HER2. The strongest comparison model in the reported table, UNI, gives $256 \times 256$9, $256 \times 256$0, and $256 \times 256$1, respectively (Kong et al., 7 Jul 2025).

For cross-modal prediction, the reported Pearson and Spearman correlations are 0.43 and 0.41. For cross-modal generation, the reported image quality metrics are PSNR $256 \times 256$2 and SSIM $256 \times 256$3. Baselines such as UNI and CONCH are listed as not directly supporting image generation in that comparison (Kong et al., 7 Jul 2025).

The ablation study isolates the contributions of reconstruction, hierarchy, and multimodal fusion. A cell-level-only model reports loss 0.405 and accuracy 0.93; adding the MAE loss gives 0.396 and 0.94; adding multi-level hierarchy gives 0.369 and 0.97; and the full model with fusion gives 0.361 and 0.98. The reported pattern assigns the largest gain to the multi-level hierarchy, with additional improvement from cross-modal fusion (Kong et al., 7 Jul 2025).

Qualitatively, generated cell images are reported as realistic at single-cell resolution and responsive to transcriptomic perturbations. This suggests that the learned representation captures a usable correspondence between gene-expression variation and morphological variation, rather than only supporting discriminative downstream tasks.

5. Relation to adjacent spatially aware modeling

SPATIA belongs to a wider methodological trend in which spatial structure is built directly into learning systems rather than treated as a secondary feature. In spatial omics, the statistical framework presented in “pasta” emphasizes that imaging-based assays can be represented as point patterns or irregular lattices and sequencing-based assays as regular lattices, with explicit attention to homogeneity, edge effects, neighbor definitions, and scale (Emons et al., 2024). SPATIA is complementary to that framework: it is a transformer-based predictive and generative model, whereas “pasta” systematizes the use of spatial statistics such as K/L functions, Moran’s I, and local indicators of spatial association.

Comparable design choices appear in other spatial domains. SPATE-GAN introduces a spatio-temporal association metric as an embedding for generative modeling of turbulent flows, log-Gaussian Cox processes, and weather fields (Klemmer et al., 2021). SPATIAL uses unidirectional temporal LSTMs within sensors and bidirectional spatial LSTMs across sensors for environmental forecasting (Hu et al., 2021). SpaRRTa evaluates whether frozen visual foundation models encode abstract, relational 3D structure between objects (Kargin et al., 16 Jan 2026). SPUR injects First-Order Ambisonics features into large audio-LLMs to support direction, elevation, distance, and overlap reasoning (Sakshi et al., 10 Nov 2025). Spatia for video generation preserves a 3D scene point cloud as persistent spatial memory and updates it through visual SLAM (Zhao et al., 17 Dec 2025). Spadas unifies dataset-level and point-level search over spatial repositories (Yang et al., 2024). The spherical spatial autoregressive model extends SAR-style dependence to responses on unit spheres in finite or infinite-dimensional Hilbert spaces (Xu et al., 23 Jan 2026). TPiet-QL extends SOLAP to temporal queries over spatial objects that split, merge, or otherwise change discretely over time (Bisceglia et al., 2011).

This suggests that “spatial awareness” now denotes several distinct but related operations: multiscale cellular context in omics, space-time dependence in environmental fields, relational viewpoint reasoning in vision, listener-centric geometry in audio, persistent scene memory in video, and unified indexing or autoregression in spatial statistics and databases. SPATIA’s distinctive contribution within that landscape is the joint modeling of morphology, transcriptomics, and biological spatial context at cell, niche, and tissue scales (Kong et al., 7 Jul 2025).

6. Limitations and prospective extensions

The reported limitations are largely tied to data regime, geometry, and interpretability. SPATIA relies on very large paired image-gene datasets, is presently formulated for 2D single-section data, defines niches as fixed non-overlapping $256 \times 256$4 pixel tiles, and is trained on Xenium-like image-based spatial transcriptomics, so transfer to MERFISH, CosMx, or Visium HD may require adaptation or fine-tuning. The paper also notes that more systematic interpretability is needed to relate specific genes and morphological features to model outputs (Kong et al., 7 Jul 2025).

The reported future directions include extending the framework to additional spatial omics modalities such as proteins and chromatin accessibility, incorporating temporal dynamics, scaling to larger cohorts and clinical applications, improving niche and tissue modeling with explicit graphs or 3D context, and using the generative module for virtual perturbation screens and in silico experiments (Kong et al., 7 Jul 2025).

A plausible implication is that SPATIA is best understood not as a closed architecture for a single benchmark, but as a template for biologically grounded spatial representation learning: modality-specific encoders, cross-attention fusion, hierarchical context aggregation, and conditional generation are all treated as components that can be replaced or extended as spatial omics technologies evolve.