Virtual Staining in Digital Histology

• Virtual staining is a technique that uses deep learning to digitally generate stained histology images from alternative microscopy inputs, eliminating the need for chemical staining.
• It overcomes limitations of conventional staining by reducing time, cost, and tissue damage while enabling multiplex analysis across various imaging modalities.
• Recent models, including GANs, CycleGANs, and diffusion methods, ensure high structural fidelity and robust compatibility with downstream diagnostic tasks such as segmentation and morphological profiling.

Virtual staining is the use of deep learning to digitally generate the appearance of a histological stain from an alternative microscopy input, or to transform one staining type into another, without performing the corresponding chemical staining protocol. In microscopy-oriented work, the same idea is also described as in silico labeling: a model predicts fluorescence-like images from label-free inputs such as transmitted-light microscopy, brightfield, phase contrast, or autofluorescence. Across pathology and bioimaging, the motivation is consistent: conventional staining is time-consuming, labor-intensive, costly, destructive to tissue, and often difficult to multiplex on the same section. At the same time, successful virtual staining is not merely a style-transfer problem, because the generated image is frequently used for diagnosis, segmentation, morphological profiling, or other downstream biological analysis (Bai et al., 2022, Kalinin et al., 7 Jul 2025).

1. Conceptual scope and source–target modalities

The literature distinguishes two broad classes of virtual staining. In label-free virtual staining, the source image is unstained or label-free and the target is a stained appearance. In stain-to-stain transformation, the source is already stained tissue and the model predicts another stain domain. The review literature explicitly groups the field in this way and documents source modalities including autofluorescence microscopy, quantitative phase imaging, nonlinear multiphoton imaging, bright-field imaging of unstained tissue, ultraviolet imaging, ultraviolet photoacoustic microscopy, total-absorption photoacoustic remote sensing, reflectance confocal microscopy, holographic microscopy, oblique back-illumination microscopy, and FT-IR spectroscopy; target domains include H&E, Masson’s trichrome, Jones silver, PAS, PASM, EVG, PSR, Orcein, IHC markers, and IF markers (Bai et al., 2022).

Recent studies show that the operational definition has widened beyond conventional 2D optical histology. In 3D cell microscopy, virtual staining predicts Hoechst-labeled nuclei from label-free transmitted-light microscopy and is evaluated by segmentation and morphological profiling rather than visual plausibility alone (Kalinin et al., 7 Jul 2025). In digital pathology, virtual stain transfer can map H&E-stained kidney biopsy images into virtual PAS stain through a cascaded framework trained with autofluorescence as a bridge (Yang et al., 2022). In imaging mass spectrometry, a diffusion model generates PAS-like brightfield images directly from ion images of label-free kidney tissue, simultaneously adding histology-like morphology and increasing spatial resolution (Zhang et al., 2024). In 3D X-ray histology, a modified CycleGAN translates synchrotron micro-CT slices into Toluidine Blue–Pyronine Y stained histology appearance and is then applied slice-by-slice to full CT volumes (Irvine et al., 11 Sep 2025).

Source domain	Virtual target	Representative setting
Label-free transmitted-light / autofluorescence / phase / holography	Fluorescence-like images or brightfield histology	3D nuclei, H&E, MT, EVG, PAS, sperm-cell staining (Kalinin et al., 7 Jul 2025, Li et al., 2024, Nygate et al., 2019)
H&E or other histochemical stains	Special stains, IHC, IF, or stain normalization	PAS, Ki67, CK8/18, HER2, PR, ER (Yang et al., 2022, Kataria et al., 2024, Saurav et al., 13 Mar 2026)
IMS or SR-µCT	PAS-like or toluidine-blue-like histology	Kidney IMS, 3D bone-implant X-ray histology (Zhang et al., 2024, Irvine et al., 11 Sep 2025)

2. Learning formulations and model families

At the formulation level, virtual staining is usually cast as image-to-image translation. In supervised settings, the generator learns a mapping $G:x \rightarrow y$, where $x$ is the source modality and $y$ is the target stain. In unpaired settings, CycleGAN-style formulations introduce forward and reverse mappings with cycle consistency, written in the review as $x^* = F(G(x))$ and $y^* = G(F(y))$, to preserve content without requiring perfectly registered pairs (Bai et al., 2022).

Early and still common implementations are GAN-based. U-Net-like generators, ResNet generators, PatchGAN discriminators, and structurally conditioned GAN frameworks recur across pathology and microscopy. An early unsupervised example used CycleGAN to generate virtual FAP-CK immunohistochemistry from real Ki67-CD8 IHC and also analyzed the effect of tiling artifacts caused by normalization layers (Lahiani et al., 2018). A supervised stain-transfer example is the cascaded deep neural network (C-DNN) for H&E $\rightarrow$ PAS, in which training proceeds through autofluorescence $\rightarrow$ H&E $\rightarrow$ PAS, while test-time deployment uses only the second generator for blind H&E-to-PAS transfer (Yang et al., 2022).

Diffusion-based formulations have become prominent in settings where realism, stochastic generative capacity, or extreme resolution enhancement are central. StainDiffuser trains two diffusion processes simultaneously—virtual IHC generation and H&E-based cell segmentation—so that the auxiliary segmentation task regularizes the model toward biologically relevant cells under limited data (Kataria et al., 2024). Brownian bridge diffusion models have been used for conditional virtual staining from low-resolution autofluorescence to H&E-like outputs with super-resolution, and also for direct translation from imaging mass spectrometry to PAS-like histology (Zhang et al., 2024, Zhang et al., 2024). Prompt- and token-conditioned models extend this further: VIMs uses text prompts and LoRA-adapted latent diffusion to generate multiple IHC stains from a single H&E image, while UNIStainNet conditions a SPADE-UNet on dense spatial tokens from the frozen pathology foundation model UNI and uses learned stain embeddings so that one unified model can serve HER2, Ki67, ER, and PR (Dubey et al., 2024, Saurav et al., 13 Mar 2026).

3. Structural fidelity, alignment, scale, and stability

A persistent technical theme is that pathology and microscopy require strict preservation of geometry. Multiple papers state explicitly that virtual staining is not generic visual translation: slight structural alterations can change diagnostic semantic information, and in 3D microscopy the predicted image must preserve cell shape, boundaries, and geometry for segmentation and profiling (Xiong et al., 2024, Kalinin et al., 7 Jul 2025).

This requirement makes registration and misalignment central. Supervised methods need paired data, yet staining is destructive, adjacent sections are not pixelwise aligned, and chemical processing can warp tissue. In autopsy virtual H&E, the RegiStain framework jointly trains a virtual staining generator, discriminator, and registration analysis network so that registration happens during training rather than through precomputed fully elastic alignment of a very large dataset (Li et al., 2023). In histopathology, DGR (Decoupled Generation and Registration) separates registration used to correct noisy supervision from registration used to enforce position consistency of the generator output, addressing the case where paired slides are only roughly matched (Ma et al., 17 Sep 2025). In transplant biopsy virtual staining, the training data were prepared through whole-slide rigid registration, patch-level affine registration, and iterative elastic pyramid cross-correlation registration, repeated 3–5 cycles until pixel-level matching was achieved (Li et al., 2024).

Another thread concerns foreground awareness. The 3D nuclear virtual-staining method Spotlight argues that training with MSE or MAE over the full volume overweights biologically irrelevant background voxels, especially in 3D. It therefore estimates a foreground mask $M$ from the target fluorescence image using Otsu’s method, applies a masked pixel-wise regression term $L_{\mathrm{MMSE}}$, adds a Dice loss on soft-thresholded predictions, and combines them as $x$0 with $x$1 (Kalinin et al., 7 Jul 2025). The result is a shape-aware objective intended to reduce axial elongation, blur, and background artifacts.

Whole-slide and large-field inference introduces additional constraints. Patch-wise inference with instance normalization can create visible seams because tile-dependent mean and variance statistics differ across neighboring regions; an early CycleGAN pathology study addressed this with overlapping tiles and a sliding-window normalization strategy (Lahiani et al., 2018). A later large-scale framework, VM-GAN, replaced identity loss with a value mapping constraint in HSV space and added a confidence-based tiling method that weights the center of each patch more strongly than the edges during recomposition (Wang et al., 7 Jan 2025).

Acquisition conditions can also dominate system design. A cascaded Deep-R refocusing network followed by a virtual staining network was introduced to stain defocused autofluorescence images of unlabeled tissue, reducing the need for precise autofocus while preserving final H&E quality (Zhang et al., 2022). Diffusion models add a different stability problem—run-to-run stochastic variance. Both the IMS and autofluorescence super-resolution studies proposed engineered sampling strategies, especially mean sampling, to reduce variance during inference without retraining (Zhang et al., 2024, Zhang et al., 2024).

4. Application domains

The application space is unusually broad, spanning cell biology, clinical pathology, molecular imaging, and volumetric histology.

In 3D cellular microscopy, virtual staining has been used to predict fluorescence-like nuclei from label-free transmitted-light stacks. Spotlight was evaluated on the DNA subset of the Allen Institute for Cell Science label-free dataset and was designed specifically so that the virtual stain would be better suited for downstream segmentation and morphological profiling rather than only pixel-level reconstruction (Kalinin et al., 7 Jul 2025).

In digital pathology, a major use case is generation of special stains or marker-specific images from routine slides. Kidney applications include unsupervised virtual IHC generation from one stain domain to another (Lahiani et al., 2018), cascaded H&E-to-PAS transfer (Yang et al., 2022), and H&E-to-IHC diffusion models such as StainDiffuser and UNIStainNet (Kataria et al., 2024, Saurav et al., 13 Mar 2026). Breast pathology includes H&E $x$2 PHH3 translation for mitosis detection, where GAN-generated synthetic images or GAN feature maps were used to train mitosis classifiers (Mercan et al., 2020). Text-conditioned multiplex IHC generation from a single H&E section has also been demonstrated for CDX2 and CK8/18 using a single diffusion model trained only on uniplex pairs (Dubey et al., 2024).

Clinical deployment-oriented studies are increasingly common. For lung and heart transplant biopsies, neural networks generated virtual H&E, MT, and EVG for lung and virtual H&E and MT for heart from label-free autofluorescence; blind pathologist evaluation reported diagnostic concordance of 82.4% for lung and 91.7% for heart relative to histochemical slides (Li et al., 2024). For autopsy lung tissue, virtual H&E from unlabeled autofluorescence was proposed as a way to bypass autolysis-induced histochemical artifacts, using RegiStain to exploit a large imperfectly registered training corpus and showing generalization to unseen COVID-19 cases (Li et al., 2023).

Beyond conventional brightfield or fluorescence, recent work extends virtual staining to other acquisition physics. In holographic microscopy, HoloStain converts stain-free digital holograms of sperm cells into virtually stained images resembling chemically stained bright-field microscopy, with blinded embryologist analysis showing a 5-fold recall improvement relative to stain-free bright-field images (Nygate et al., 2019). In imaging mass spectrometry, a Brownian bridge diffusion model generates PAS-like images from 1,453 selected m/z channels of human kidney IMS data, enabling histology-like interpretation from intrinsically label-free molecular maps (Zhang et al., 2024). In 3D X-ray histology, virtually stained synchrotron micro-CT volumes of bone implants provide histology-like semantic color while preserving volumetric, non-destructive acquisition (Irvine et al., 11 Sep 2025).

5. Evaluation, downstream utility, and recurring misconceptions

The field uses a heterogeneous evaluation stack. When paired data exist, standard image-level metrics include SSIM, PSNR, MS-SSIM, MSE, MAE, LPIPS, FID, KID, and related feature-space distances; unpaired settings often rely more heavily on FID-like or pathologist-based assessments (Bai et al., 2022, Kataria et al., 2024). Yet a recurring conclusion is that image similarity alone is insufficient.

One misconception addressed explicitly in several papers is that virtual staining can be validated as if it were ordinary natural-image synthesis. The 3D nuclei work argues that a prediction can look numerically good while reproducing background noise, blur, or axial artifacts that make nuclei harder to segment and distort morphology measurements (Kalinin et al., 7 Jul 2025). The transplant-biopsy study therefore combined whole-slide visual review with blinded diagnostic voting by board-certified pathologists (Li et al., 2024). UNIStainNet further emphasized that SSIM/PSNR are not reliable for virtual staining because of ground-truth misalignment, and reported distributional metrics together with stain-quantification measures such as Pearson-$x$3 on DAB intensity and DAB KL divergence (Saurav et al., 13 Mar 2026).

Another misconception is that a visually faithful virtual stain is automatically useful for downstream models. A systematic study of segmentation and classification from label-free, virtually stained, and ground-truth fluorescence images found that the utility of virtual staining depends strongly on the capacity of the task network. For low-capacity segmentation or classification networks, virtual staining could improve Dice or AUC by converting the input into a more task-friendly representation. For sufficiently high-capacity task networks, label-free input performed about as well as, or better than, the virtually stained input; in some classification cases virtual staining degraded performance. The paper interpreted this through the data processing inequality, arguing that virtual staining can help suboptimal learners without creating new task-relevant information (Sengupta et al., 31 Jul 2025).

Downstream validation therefore has become central. Examples include watershed and Cellpose segmentation on predicted 3D nuclei (Kalinin et al., 7 Jul 2025), mitosis detection using synthetic PHH3/H&E translations or GAN feature maps (Mercan et al., 2020), gland segmentation on generated IHC images in VIMs (Dubey et al., 2024), and glomerular detection or segmentation improvements after unpaired renal virtual staining with VPGAN/HARBOR (Chen et al., 22 Apr 2025). The evaluation trend is clear: pathology-oriented virtual staining is increasingly judged by analytical usefulness, diagnostic concordance, and structural correctness rather than by visual realism alone.

6. Limitations, controversies, and research directions

The most persistent limitation is that the ability to simulate a stain is constrained by what information is actually present in the input domain. An early unsupervised IHC study found much stronger agreement for CK$x$4 density than for FAP density and explicitly interpreted the weaker FAP result as a biological limitation: if a target is not directly visible in the source stain, the model cannot reliably infer it (Lahiani et al., 2018). This concern reappears in later work on difficult IHC markers, misaligned consecutive sections, and stain quantification.

A second limitation is generalization under real clinical variability. The review literature emphasizes ground-truth variability, scanner and laboratory differences, difficult registration, hallucination risk, limited standardization, and the need for broad multi-institution validation before routine clinical adoption (Bai et al., 2022). Clinical studies echo this cautiously. The transplant-biopsy work notes modest cohort sizes, exclusion of slides with severe artifacts or insufficient fragments, and the need for broader validation across more samples and rejection stages (Li et al., 2024). The IMS PAS study was demonstrated on kidney tissue from 5 patients and explicitly notes dependence on registered training pairs and tissue scope (Zhang et al., 2024).

A third limitation concerns error concentration rather than average quality. UNIStainNet showed that failures were systematic rather than random, concentrating in non-tumor tissue, especially adipose, necrosis, and background, while invasive carcinoma had relatively low failure rates (Saurav et al., 13 Mar 2026). This kind of tissue-stratified analysis has become important because mean FID or SSIM can obscure clinically significant local failures.

Current research directions follow directly from these constraints. Foundation-model and VLM guidance are being used to inject pathology knowledge into the generator itself or into auxiliary prompt spaces, as in UNI-conditioned multi-stain generation and VLM-guided unpaired translation with concept anchors (Saurav et al., 13 Mar 2026, Chen et al., 22 Apr 2025). Misalignment-aware objectives and cascaded registration continue to address the scarcity of perfectly paired data (Ma et al., 17 Sep 2025). Diffusion-based methods are being adapted for higher resolution, lower variance, and more stable inference (Zhang et al., 2024). At the same time, review work identifies broader needs: standardized large-scale datasets, higher-throughput label-free imaging, better generalization, task-specific loss functions, automated evaluation tools, validation on fresh tissue and intraoperative settings, and extensions beyond conventional histology toward marker inference and other computationally generated contrasts (Bai et al., 2022).