Gene-DML: Dual-Pathway & Diffusion Models

• Gene-DML is a unified framework that combines dual-pathway multi-level discrimination for histopathological gene expression prediction with a gene-level masked diffusion model for DNA sequence generation.
• It aligns multi-scale image embeddings with gene vectors using contrastive and clustering techniques, effectively capturing tissue architecture and gene regulatory nuances.
• Empirical evaluations show significant performance gains over prior methods, demonstrating improved prediction accuracy and robust bidirectional sequence modeling.

Gene-DML refers to two distinct, state-of-the-art frameworks at the intersection of computational genomics and machine learning: (1) a dual-pathway multi-level discrimination method for predicting gene expression from histopathology images (Song et al., 19 Jul 2025), and (2) a gene-level masked diffusion LLM for bidirectional sequence modeling and generation, building upon discrete DNA diffusion objectives (Yang et al., 2 Mar 2026). Both are representative of broader efforts to unify cross-modal representation learning and generative modeling at gene-relevant resolutions.

1. Definition and Motivation

Gene-DML, as introduced by Chen et al., is a unified framework designed to enhance the alignment between histopathological morphology and gene expression profiles by structuring the joint latent space via dual-pathway, multi-level discrimination (Song et al., 19 Jul 2025). The approach addresses limitations in single-scale, unimodal, or instance-only contrastive learning by capturing correlates of gene regulation and tissue architecture at multiple spatial and organizational scales.

Separately, the natural extension of diffusion-based sequence models to the gene level, discussed in the context of D3LM (Discrete DNA Diffusion LLM), provides a framework for modeling entire genes (including complex regulatory and structural features) using discrete masked diffusion objectives (Yang et al., 2 Mar 2026). Such Gene-DML models enable unified bidirectional understanding and generation of gene-scale DNA sequences, accommodating gene architecture granularity.

2. Dual-Pathway Multi-Level Discrimination in Spatial Transcriptomics

Gene-DML implements two parallel discrimination pathways to align morphological and transcriptional data derived from spatial transcriptomics and histopathology:

1. Multi-Scale Instance-Level Discrimination: For each spatial transcriptomics (ST) spot, the model extracts image embeddings at three scales—local (224×224 px patch), neighbor (1120×1120 px), and global context—using a frozen, pretrained histopathology encoder. These embeddings are linearly projected to a shared d-dimensional space and aligned with gene expression embeddings via symmetric cross-entropy contrastive loss. The internal similarity target $T^s \in \mathbb{R}^{N \times N}$ is defined as

$$T^s_{ij} = \mathrm{softmax}_j\left(\frac{1}{2\tau}[E^s_i \cdot E^s_j + E^G_i \cdot E^G_j]\right)$$

yielding the instance-level loss $\mathcal{L}_{\mathrm{ins}^s}$ for each scale $s$.

2. Cross-Level Instance–Group Discrimination: The second pathway projects both image and gene embeddings to clustering subspaces, computes $k$-means centroids in each modality, and aligns instances in one modality to clusters of the other. Assignments are by nearest centroid in the opposite modality, with cross-entropy losses enforcing these cross-modal associations.

The total training loss combines the multi-scale instance-level discrimination, the instance–group discrimination (weighted by $\lambda$), and a regression objective mapping fused image embeddings to gene expression.

3. Model Architecture and Training Protocol

Gene-DML’s architecture is modular, with distinct branches for morphology and transcriptomics:

• Image Pathway: Each scale input is processed with a frozen UNI encoder, linearly projected, and then fused (via concatenation or a transformer) for downstream contrastive and regression tasks.
• Gene Pathway: A two-layer feed-forward network with GELU and dropout encodes filtered gene vectors into the shared latent space.
• Clustering Branch: Both image and gene embeddings are projected, normalized, and clustered via $k$-means, yielding centroids used in the cross-level loss.
• Regression Head: A lightweight MLP predicts gene expression vectors from fused image embeddings.

Training employs Adam optimization ($\mathrm{lr}=1\mathrm{e}{-4}$) with StepLR decay, batch size 256, latent dimension $d=512$, and $k$ clusters chosen per dataset (e.g., 25 for skinST). Cross-validation employs a patient-wise split; external generalization is evaluated on 10× Visium samples.

4. Evaluation, Empirical Gains, and Quantitative Metrics

Performance is evaluated on public ST datasets (HER2ST, STNet, skinST) with the following average results (mean ± std):

Dataset	MSE	PCC(A)	PCC(H)
HER2ST	0.210±0.03	0.331±0.07	0.541±0.08
STNet	0.179±0.02	0.237±0.09	0.384±0.08
skinST	0.237±0.04	0.433±0.03	0.520±0.07

On all datasets, Gene-DML outperforms prior methods (TRIPLEX, M2OST, Hist2ST) by 5–10 PCC points and 0.02–0.05 MSE. External generalization to Visium datasets also yields consistently higher PCC(A) and PCC(H) than the best baseline, with gains statistically significant at $$T^s_{ij} = \mathrm{softmax}_j\left(\frac{1}{2\tau}[E^s_i \cdot E^s_j + E^G_i \cdot E^G_j]\right)$$0.

5. Structural Mechanisms and Advantages

Gene-DML’s design addresses key systemic challenges:

• Scale-aware alignment: By explicitly extracting and contrasting features at the cellular, neighborhood, and tissue-architectural scales, the model captures fine and coarse morphological determinants of gene expression.
• Group-aware discrimination: Cross-modal clustering constrains the representation geometry, reducing over-separation of similar tissue spots and enforcing biological coherence in the joint embedding space.
• Improved regression: The dual-pathway contrastive objectives increase data efficiency and generalization in the final regression mapping from image to gene expression.

A plausible implication is that such structured latent representations may improve not only prediction accuracy but also the interpretability of discovered morphological-genomic relationships.

6. Diffusion-Based Gene-Level Sequence Modeling

In the orthogonal context of generative modeling, the Gene-DML extension of D3LM retools the masked discrete diffusion machinery for gene-scale DNA inputs (Yang et al., 2 Mar 2026):

• Backbone: Utilizes the Nucleotide Transformer v2 encoder (bidirectional Transformer, rotary embeddings, SwiGLU activations, 50M–500M parameters).
• Masked Diffusion Objective: Noising is performed by randomly masking tokens with probability $$T^s_{ij} = \mathrm{softmax}_j\left(\frac{1}{2\tau}[E^s_i \cdot E^s_j + E^G_i \cdot E^G_j]\right)$$1:

$$T^s_{ij} = \mathrm{softmax}_j\left(\frac{1}{2\tau}[E^s_i \cdot E^s_j + E^G_i \cdot E^G_j]\right)$$2

The objective is the cross-entropy on masked positions, reweighted by $$T^s_{ij} = \mathrm{softmax}_j\left(\frac{1}{2\tau}[E^s_i \cdot E^s_j + E^G_i \cdot E^G_j]\right)$$3.

• Tokenization: Best generative fidelity is achieved at 6-mer tokenization (vocabulary ≈ 4096+special), with potential for hybrid (e.g., codon) schemes in coding regions.
• Sampling/Generation: Generation proceeds in $$T^s_{ij} = \mathrm{softmax}_j\left(\frac{1}{2\tau}[E^s_i \cdot E^s_j + E^G_i \cdot E^G_j]\right)$$4 discrete steps, unmasking random subsets of masked positions per step. The temperature parameter $$T^s_{ij} = \mathrm{softmax}_j\left(\frac{1}{2\tau}[E^s_i \cdot E^s_j + E^G_i \cdot E^G_j]\right)$$5 optimally balances fidelity and diversity.

For gene-level modeling, extensions include hierarchical diffusion over exons/introns, task-driven masking, conditional generation given transcription factor/chromatin states, and combined inference (e.g., variant effect prediction).

7. Impact, Implications, and Research Directions

The Gene-DML frameworks represent two convergent trends at gene-level molecular modeling:

• Cross-modal embedding: Structured joint representations across image and gene modalities enable accurate, scalable, and generalizable inference of molecular states from routine histopathology (Song et al., 19 Jul 2025). This suggests wider applicability in computational pathology and integration with spatial genomics.
• Bidirectional generative modeling: Discrete masked diffusion at the gene resolution supports unified modeling of sequence understanding and conditional sequence generation, adaptable to regulatory and structural gene contexts (Yang et al., 2 Mar 2026). A plausible implication is more accurate gene circuit or element design and variant effect prediction workflows.

Both frameworks highlight the importance of architectural alignment with biological structure (multi-scale, multi-region, conditional differentiation) and show that explicit modeling of molecular heterogeneity and context yields significant empirical gains in genomics research workflows.