CRAFTS: Correlation-Regulated Tissue Synthesis

• The paper introduces a novel correlation-regulated alignment mechanism that suppresses semantic drift and morphological hallucinations in synthetic histology images.
• It employs a dual-stage training pipeline using multi-source datasets to enhance biological plausibility and improve diagnostic accuracy.
• Empirical evaluations show CRAFTS outperforms baselines in key metrics, boosting downstream clinical tasks via effective synthetic data augmentation.

The Correlation-Regulated Alignment Framework for Tissue Synthesis (CRAFTS) is a pathology-specific text-to-image generative foundation model designed to address critical limitations in computational pathology. CRAFTS is the first system of its kind capable of synthesizing clinically faithful histology images from free-form pathological descriptions, overcoming challenges of data scarcity, imbalance, and the need for extensive expert annotation. It introduces a novel alignment mechanism to suppress semantic drift and morphological hallucinations, ensuring both biological plausibility and diagnostic accuracy in generated outputs (Guan et al., 15 Dec 2025).

1. Motivation and Problem Context

The advancement of clinical-grade artificial intelligence in pathology is hindered by the limited availability of large, diverse, and high-quality annotated datasets. While generative models theoretically provide a solution, existing systems—primarily diffusion-based or GAN-based text-to-image architectures—are prone to two principal failure modes: semantic drift and morphological hallucinations. Semantic drift refers to the model's tendency to lose clinically meaningful semantics, instead producing images with superficially plausible textures that miss crucial diagnostic features such as pleomorphism or stromal invasion. Morphological hallucinations arise when models, trained with weak supervision or on noisy captions, generate non-biological structures, thereby jeopardizing the diagnostic value of synthetic images. CRAFTS directly targets these issues to expand accessible training resources for pathology AI (Guan et al., 15 Dec 2025).

2. Model Architecture and Training Pipeline

CRAFTS is anchored on a latent diffusion backbone that leverages a variational autoencoder (VAE) to encode 512×512 hematoxylin and eosin (H&E) image patches into a latent space, $z_0 = E_{VAE}(x_0)$. The system employs a U-Net-based denoiser, $\epsilon_\theta(z_t, t, c)$, which is conditioned on textual input $c$ and learns to predict noise in $z_t$. Diffusion proceeds with a linear schedule $\beta_t \in [\beta_{min}, \beta_{max}]$, and the standard forward-reverse Markov process with latent updates given by:

$$z_{t-1} = \sqrt{\alpha_t^{-1}} \bigg( z_t - \frac{\beta_t}{\sqrt{1-\bar{\alpha}_t}} \epsilon_\theta(z_t, t, c) \bigg) + \sigma_t \mathcal{N}(0, I)$$

where $\bar{\alpha}_t = \prod_{i=1}^{t} (1 - \beta_i)$. Text encoding is performed by CLIP-ViT-large (patch14), generating token embeddings $\{t_i\}_{i=1}^N$. Cross-attention layers within the U-Net enforce text-image feature alignment at multiple resolutions; lower layers focus on global patterns, while higher layers refine local, cellular texture.

Training consists of two stages:

• Stage 1 (Pre-training): Weakly supervised on approximately 1.2 million image–caption pairs from PubMedVision, PMC-OA, PMC-VQA, and Quilt-1M for coarse-grained cross-modal grounding.
• Stage 2 (Fine-tuning): Pathology-focused training on ~1.6 million TCGA H&E tiles with expert captions and 30 cancer-type labels, designed to inject disease-type priors and strengthen semantic disentanglement.

3. Correlation-Regulated Alignment Mechanism

To address the persistent challenges of semantic drift and morphological hallucinations, CRAFTS incorporates a dual-metric relational consistency loss and adaptive category-guidance during training. The alignment loss at each diffusion step is defined as:

$$\mathcal{L}_{align} = \mathcal{L}_{diffusion} + \lambda_{corr}\mathcal{L}_{corr} + \lambda_{cat}\mathcal{L}_{cat}$$

• $\mathcal{L}_{diffusion}$: Standard denoising matching objective.
• $\mathcal{L}_{corr}$: Correlation regularization. For a batch of text features $\{t_i\}_{i=1}^B$ and their paired image latents $\{z_i\}$, the method computes cosine similarity matrices for text–text ($M^T_{ij}$) and image–image ($M^I_{ij}$) pairs, minimizing the difference $(M^I_{ij} - M^T_{ij})^2$ over all pairs.
• $\mathcal{L}_{cat}$: Adaptive category-guidance. This term, used in fine-tuning, exploits per-pair weights based on the cosine similarity of text to its ground-truth cancer-type label, steering image latents towards the correct disease manifold.

This alignment mechanism enforces that semantically close prompts yield proportionally similar image latents, filtering stochastic noise and preserving diagnostic structures. Category guidance sharpens inter-class cluster boundaries, reducing confusion across similar cancer types. Notably, CRAFTS achieves a silhouette coefficient for cancer-type clustering of +34.37%, significantly higher than Stable Diffusion (13.68%) and negative for other baselines.

4. Data Sources and Implementation

CRAFTS leverages multi-source data for both stages of training:

• Pre-training datasets: PubMedVision (13.2K), PMC-OA (42K), PMC-VQA (17K), and Quilt-1M (1M), totaling around 1.2 million image–caption pairs.
• Fine-tuning dataset: 1.6 million H&E image tiles from TCGA, each with validated expert captions and one of 30 cancer-type labels (e.g., BRCA, LUAD, SKCM).

Implementation utilizes a Stable Diffusion v1.5 U-Net backbone with CLIP (ViT-large), totaling approximately 1 billion parameters, trained using 10× NVIDIA RTX A6000 GPUs. The system operates at 512×512 resolution, batch size 320, and adopts standard AdamW optimization with β₁ = 0.9, β₂ = 0.999, and weight decay $1 \times 10^{-2}$. Pre-training runs for 1.2 million steps (~2 days), fine-tuning for 1.6 million steps (~4 days), and inference uses 50 diffusion steps with a classifier-free guidance scale of 7.5.

5. Empirical Evaluation and Pathological Validation

CRAFTS performance is benchmarked against leading generative models using both objective and subjective metrics:

• PLIP-FID (Pathology-aware Fréchet Inception Distance): CRAFTS 11.32; outperforms Stable Diffusion (15.82), Imagen (21.21), and StyleGAN-T (17.81).
• PLIP-I (Image–Image Cosine Similarity): 85.74% for CRAFTS, surpassing other models (78.50–82.11%).
• PLIP-T (Text–Image Cosine Similarity): 29.24% for CRAFTS versus 25.07–28.45% for baselines.
• Pathologist study (N=3, consensus): CRAFTS images yield the lowest real-vs.-synthetic discrimination F1 (66.39%), indicating closest realism. Semantic alignment ranking (mean 3.27 on a 1–4 scale) exceeds that of baselines (2.05–2.60).

In terms of cancer-type feature separability (t-SNE + silhouette), CRAFTS’ clustering by cancer type is robust (+34.37%), whereas baselines fail to deliver clear separation.

6. Augmentation and Downstream Task Performance

CRAFTS-generated synthetic data augments real data in multiple downstream clinical tasks, exhibiting consistent performance improvements at synthetic:real ratios up to 10:1. The following summarizes performance enhancements:

Task	Baseline (Real)	+CRAFTS Synthetic
BACH Classification (4-class)	44.73%	+5.38 pp to 50.11%
BRACS (7-class)	50.88%	+7.58 pp to 58.46%
BreakHis	56.10%	60.33%
LungHist	44.96%	53.25%
ARCH Cross-modal T→I	32.65%	35.92%
ARCH Cross-modal I→T	30.22%	35.44%

Self-supervised learning (SimSiam) and visual question answering tasks (PatchVQA, ARCH) also show marked gains. For instance, PatchVQA METEOR increases from 9.50% to 16.60% with CRAFTS augmentation. This demonstrates the synthetic data’s efficacy in supporting discriminative, cross-modal, and grounded learning.

7. Conditional Control and Phenotypic Precision

CRAFTS integrates ControlNet to facilitate direct conditioning on structural or phenotype prompts, enhancing generation precision for tissue architecture:

• Nuclear Segmentation masks (GLySAC, CoNSeP): CRAFTS achieves improved PLIP-FID, SSIM, and MSE relative to Stable Diffusion, and exhibits accurate preservation of nuclear geometry and gland lumina.
• Immunofluorescence-to-H&E tasks (HEMIT, SHIFT): CRAFTS delivers higher SSIM and NCC with lower PLIP-FID and MSE, faithfully translating fluorescence signal gradients into histological staining variations.

These evaluations substantiate the model's capacity for user-driven, restriction-respecting synthetic histology, supporting advanced applications in diagnostic tool development where precise architectural or marker expression constraints are required.

8. Applications, Limitations, and Future Prospects

CRAFTS enables several application scenarios: generation of balanced datasets for rare phenotypes, privacy-preserving data sharing by substituting synthetic for patient-derived slides, and controlled synthesis for targeted clinical investigation (e.g., spindle-cell sarcoma from few-shot prompts).

Current limitations include restriction to 512×512 patch-level synthesis, with whole-slide imaging at gigapixel scale considered an open challenge. The model’s dependence on CLIP’s text encoder constrains its handling of very lengthy or highly structured clinical narratives.

Anticipated directions include the integration of hierarchical or cascaded diffusion (e.g., ZoomLDM) for multi-scale whole-slide imaging, the adoption of domain-specific LLMs for richer clinical text conditioning, and the fusion of histology with structured molecular or immuno-profile data for true multi-modal tissue synthesis.

CRAFTS establishes a pathology-grounded generative foundation model that unites large-scale training, correlation-regulated alignment, and explicit disease priors to generate diagnostic-grade synthetic images. By enhancing the performance of downstream AI tasks and enabling controlled, privacy-preserving data generation, it offers a robust platform for future computational pathology research and clinical translation (Guan et al., 15 Dec 2025).