Anatomically-Informed GANs

• Anatomically-Informed GANs are specialized models that integrate anatomical constraints via segmentation priors, 3D meshes, or graph encodings to ensure structural consistency in synthetic images.
• They employ tailored architectures like mask-conditioned synthesis, subvolume anchoring, and FiLM-based modulation to generate high-fidelity biomedical imagery and realistic human renderings.
• Quantitative evaluations show significant improvements, such as FID reductions (e.g., 0.008±0.003) and increased Dice coefficients, highlighting their potential in clinical and computational pathology applications.

Anatomically-Informed GANs are a class of generative adversarial networks that explicitly encode, preserve, or are constrained by anatomical structure during synthesis, translation, or manipulation of biomedical or biological images. The distinguishing feature is the integration of priors or supervision grounded in physical, morphological, or segmentation-defined tissue, organ, or body-part information—either as explicit inputs, through architectural design, or as loss functions. These models support critical tasks in high-fidelity medical imaging, realistic human rendering, label–image co-generation, and anatomically-targeted data augmentation, outperforming unconstrained counterparts in spatial coherence, realism, and downstream task utility.

1. Defining Anatomically-Informed Generative Adversarial Networks

Anatomically-informed GANs (AIGANs) are a subset of GAN-based generative models characterized by their explicit capture, enforcement, or preservation of spatial, morphological, or structural properties corresponding to biological anatomy. Unlike generic GANs, which primarily learn appearance distributions, AIGANs introduce domain-informed priors, conditioning schemes, feature fusions, or anatomical consistency losses to propagate ground-truth structure into generated data. The anatomical information may take the form of semantic segmentations (e.g., membrane/mask maps in electron microscopy (Han et al., 2020), organ masks in PET (Guo et al., 2024, Guo et al., 2023)), 3D mesh priors (e.g., SMPL/FLAME in human generation (Yang et al., 2022, Bergman et al., 2022)), filamentary networks (Zhao et al., 2017), sketch guidance (Zhang et al., 2019), or attention to specific regions (e.g., facial Action Units (Pumarola et al., 2018)). These constraints enable models to produce anatomically plausible variations and support applications where spatial and biological fidelity is essential.

2. Architectural Strategies and Conditioning Mechanisms

AIGANs employ diverse, modality-specific architectural mechanisms to encode or enforce anatomy:

• Mask-Conditioned Synthesis: Conditioning both the generator and discriminator on semantic masks ensures precise structure (e.g., Fila-GAN for vascular topology (Zhao et al., 2017); AnatoMaskGAN for inter-slice contextual aggregation (Wu et al., 15 Aug 2025); two-stage EM GAN for membrane/mitochondrion label–image pairing (Han et al., 2020)).
• Subvolume Anchoring in 3D: Hierarchical Amortized GAN (HA-GAN) (Sun et al., 2020) amortizes GPU memory by separately generating (and backpropagating) low-res full volumes and high-res subvolumes from a shared backbone, maintaining global coherence by anchoring high-res patches to a single low-res anatomical layout.
• Graph- or GNN-Based Spatial Encoding: Slice-feature fusion via GNNs, as in AnatoMaskGAN (Wu et al., 15 Aug 2025), models inter-slice dependency to ensure 3D topological continuity and prevent spatial discontinuities between adjacent context windows.
• Sketch-Rendering Pipelines: SkrGAN (Zhang et al., 2019) mimics the human drawing process with a two-stage pipeline (sketch-generation, color rendering), enforcing structural correctness at the sketch level with a separate multi-scale discriminator.
• 3D Priors and Mesh-Driven Warping: Human image synthesis models inject 3D mesh or mesh-inferred pose as priors. 3DHumanGAN (Yang et al., 2022) uses an implicit function of a 3D posed mesh to generate spatially adaptive styles, which are then injected into a 2D StyleGAN-like backbone, while GNARF (Bergman et al., 2022) factors out pose via an explicit mesh-surface–driven deformation.
• ROI and Segmentation-Driven Modulation: TAI-GAN (Guo et al., 2024, Guo et al., 2023) augments frame input with rough cardiac segmentations and injects tracer kinetics using FiLM layers, thereby focusing translation/training on anatomical regions of interest.
• Depth and Geometry Constraints for Domain Transfer: BronchoGAN (Soliman et al., 2 Jul 2025) employs foundation model-generated depth images and explicit orifice segmentation as invariant, cross-domain anatomical anchors for high-fidelity bronchoscopy domain transfer. Dice loss between input and synthesized (re-inferred) organ segmentations ensures anatomical consistency.
• Attention Mechanisms on Anatomical Regions: GANimation (Pumarola et al., 2018) uses an attention mask to modulate color transformation only in regions corresponding to activated Action Units, supporting anatomically meaningful facial expression editing.

3. Mathematical Objectives and Anatomical Losses

AIGANs routinely combine standard adversarial losses with anatomical consistency or reconstruction losses. Representative formulations include:

• Patch and Segmentation Consistency: Discriminators operate on image/label pairs at multiple scales (e.g., (Han et al., 2020, Wu et al., 15 Aug 2025)), penalizing local and global divergence from anatomical references.
• Dice and Region Overlap Metrics: BronchoGAN uses a Dice loss on binary masks between input and depth-inferred segmentations of synthesized images to guarantee preservation of bronchial orifices (Soliman et al., 2 Jul 2025).
• Feature- or Label-Space Cycle Losses: Two-stage EM GANs incorporate cycle-consistency and segmentation reconstruction losses—if $F_y$ denotes a segmenter:

$$\lambda_{\rm cyc} \left( \mathbb{E}_{y}\,[L_{\rm CE}(y, F_y(G_x(y)))] + \mathbb{E}_{z}\,[L_{\rm CE}(G_y(z),\, F_y\circ G_x\circ G_y(z))]\right)$$

encouraging that synthetic images can be accurately reverse-segmented to match original labels (Han et al., 2020).

• Spatial Feature Fusion: Employing message-passing in GNNs enables the feature vectors of neighboring anatomical slices to be contextually fused before decoding (Wu et al., 15 Aug 2025).
• FiLM/Adaptive Normalization Modulation: Anatomy-conditioned FiLM layers ($\mathrm{FiLM}(X_i) = \gamma_i X_i + \beta_i$) modulate the generator's features by properties of ROI masks and tracer kinetics (Guo et al., 2024, Guo et al., 2023).
• Perceptual/Statistical Classifier Losses: Secondary classifier-based penalties on image texture/gray-distribution (e.g., G-TC loss in AnatoMaskGAN) help generated regions match radiological statistics within known anatomical classes (Wu et al., 15 Aug 2025).

4. Quantitative Assessments of Anatomical Fidelity

AIGANs achieve substantial, measurable improvements in anatomical coherence and downstream performance relative to unconstrained or patch-based GANs.

• Global Realism and Structural Consistency: On 256³ thorax CT, HA-GAN achieves FID=0.008±0.003 and MMD=0.022±0.010, far surpassing StyleGAN2 (FID=0.081, MMD=0.225) and eliminating sub-patch boundary artifacts (Sun et al., 2020).
• Semantic Overlap in Generation: BronchoGAN achieves a Dice coefficient of 0.6743 for bronchial orifices on ex-vivo test frames, compared to 0.2412 for unpaired cycleGAN (Soliman et al., 2 Jul 2025). Anatomical loss increases Dice by up to 0.43 on synthetic images.
• Perceptual and Downstream Task Utility: AnatoMaskGAN increases PSNR by 0.43 dB on brain MRI and SSIM by 0.48 points on abdomen CT compared to prior models; ablations confirm independent contributions from graph fusion, 3D noise, and per-class gray-texture losses (Wu et al., 15 Aug 2025).
• Segmentation and Quantification Performance: GAN-generated electron micrographs trained with anatomical supervision support U-Net segmentation to yield mIoU=89.3% and NLL=0.108 (Han et al., 2020). In PET, TAI-GAN conversion improves motion-correction error (down to 3.48 mm from 4.45 mm) and reduces myocardial blood flow estimation bias (from –36.99% down to –7.95%) (Guo et al., 2023).
• Controlled Editing and Plausibility: GANimation enables smooth, AU-conditioned, muscle-realistic facial edits with AU intensity errors below 0.05 and identity preservation within 5% (Pumarola et al., 2018).

5. Modalities, Data Preparation, and Application Domains

AIGANs are implemented across a variety of imaging modalities and structural domains:

Paper / Model	Modality / Domain	Anatomical Prior / Input
HA-GAN (Sun et al., 2020)	CT, MRI (3D)	Subvolume anchoring w/ low-res context
SkrGAN (Zhang et al., 2019)	Retina, X-Ray, MRI	Sketch prior (vessel/skeleton)
AnatoMaskGAN (Wu et al., 15 Aug 2025)	MRI, CT (Slices)	Semantic mask stack, GNN fusion
3DHumanGAN (Yang et al., 2022)	Human RGB Images	3D mesh, pose-conditioned styling
GNARF (Bergman et al., 2022)	Human RT/NeRF	Mesh-driven deformation field
TAI-GAN (Guo et al., 2024, Guo et al., 2023)	PET	ROI masks, FiLM tracer kinetics
BronchoGAN (Soliman et al., 2 Jul 2025)	Bronchoscopy	Depth + orifice segment (foundation)
EM GAN (Han et al., 2020)	Electron Microscope	Membrane/mitochondria labels
Fila-GAN (Zhao et al., 2017)	Retina, Neuron	Binary filament map
GANimation (Pumarola et al., 2018)	Face Images	AU vector (Action Unit)

Applications include high-resolution volumetric data generation, robust domain adaptation/translation, data augmentation for classification/segmentation, image-to-label paired synthesis, dynamic frame conversion, and anatomically-aware image manipulation.

6. Limitations, Extensions, and Generalization

Several limitations arise in AIGANs due to the specificity and availability of anatomical priors:

• Models such as TAI-GAN require clinician-provided segmentations, constraining automation and generalizability (Guo et al., 2024, Guo et al., 2023). A plausible implication is that future efforts will focus on automatic, weakly supervised, or foundation model-based segmentation.
• 3DGANs anchored by mesh priors (e.g., GNARF, 3DHumanGAN) inherit expressiveness and limitations of the parametric mesh model (e.g., SMPL/FLAME), potentially failing in the presence of topological changes or non-canonical anatomy (Bergman et al., 2022, Yang et al., 2022).
• Quantitative anatomical realism is domain- and metric-specific; for example, patch-level matching does not guarantee cross-view or cross-pose consistency unless enforced via explicit mesh- or graph-based priors.
• Annotation, mask, or sketch reliance may restrict applicability in unlabeled or novel imaging domains. However, approaches such as BronchoGAN, leveraging foundation depth models and training-free segmenters, suggest anatomically-informed pipelines can be made domain-agnostic (Soliman et al., 2 Jul 2025).
• Future directions point to integrating richer physical or physiological priors (e.g., as-rigid-as-possible, biomechanical constraints), leveraging diffusion or neural-ODE backbones, and generalizing constraint strategies to other modalities, including multimodal volume translation (Guo et al., 2024, Guo et al., 2023).

7. Impact and Significance across Medical and Biological Imaging

Anatomically-informed GANs have advanced the fidelity, usability, and interpretability of synthetic data in various high-stakes tasks. Integration of explicit anatomy has enabled:

• Reductions in artifact rates and improved spatial continuity of 3D biomedical reconstructions (Sun et al., 2020, Wu et al., 15 Aug 2025).
• Authentic domain transfer across physical, simulated, and artistic imaging modalities by maintaining structure invariants (Soliman et al., 2 Jul 2025, Zhang et al., 2019).
• Accurate simulation of clinical phenomena such as disease progression or tracer dynamics, with direct gains in model robustness and clinical quantification (Guo et al., 2024, Guo et al., 2023).
• Extended manipulations of human appearance (pose, expression) in a physically justifiable manifold (Yang et al., 2022, Bergman et al., 2022, Pumarola et al., 2018).
• Augmentation strategies yielding improvements in supervised learning and segmentation performance, particularly in extremely small data regimes (Zhao et al., 2017, Han et al., 2020).

AIGANs thus represent a robust paradigm for anatomically consistent image synthesis and translation, underpinning multiple applications in biomedical research, computational pathology, and clinical AI.