AnatoMaskGAN: 3D Medical Image Synthesis

• The paper introduces a novel semantic-mask-conditioned GAN that fuses multi-slice features with a GNN-driven approach to ensure anatomical continuity.
• It employs a 3D spatial noise injection strategy to deliver inter-slice stochastic consistency and enhance textural realism in generated images.
• The framework uses a grayscale-texture classifier for class-wise intensity regulation, achieving improved PSNR, SSIM, and LPIPS across datasets.

Searching arXiv for the named method and closely related mask-guided GAN work. AnatoMaskGAN most directly denotes the semantic-mask-conditioned GAN introduced in "AnatoMaskGAN: GNN-Driven Slice Feature Fusion and Noise Augmentation for Medical Semantic Image Synthesis" (Wu et al., 15 Aug 2025). In that formulation, the model synthesizes realistic grayscale medical images from contiguous stacks of 2D semantic-mask slices while explicitly modeling inter-slice spatial correlations, three-dimensional stochastic variation, and class-wise grayscale-texture statistics. In adjacent literature, the same label has also been used informally to describe anatomy-aware, mask-guided generative frameworks outside this exact paper, but those earlier systems were published under different names and problem settings (Gu et al., 2019, Abhishek et al., 2019, Phan et al., 2023, Wang, 17 Jun 2025).

1. Terminology and problem setting

The 2025 AnatoMaskGAN framework addresses a limitation common to GAN-based medical semantic synthesis: most models take a single 2D mask and produce one synthetic slice, which neglects the spatial relationships between adjacent slices in a 3D volume (Wu et al., 15 Aug 2025). The paper identifies three consequences of that one-to-one setting: structural discontinuities across slices, blurry boundaries with missing fine anatomical details, and limited structural and textural diversity because noise is injected independently per slice.

The model is defined for semantic-mask-conditioned synthesis on multi-slice scans. It operates on a contiguous stack of 2D mask slices $\{\mathrm{Slice}_i\}_{i=1}^{n}$ extracted from 3D volumes and outputs a corresponding stack of grayscale image slices $\{\mathrm{Slice}_i^G\}_{i=1}^{n}$ (Wu et al., 15 Aug 2025). Two datasets anchor the experimental setting: L2R-OASIS, with 416 T1-weighted brain MRI volumes and 35 labeled brain structures, and L2R-Abdomen CT, with 50 3D abdominal CT volumes and annotations for 13 abdominal organs (Wu et al., 15 Aug 2025). The intended use is both image synthesis and data augmentation.

This framing places AnatoMaskGAN in the semantic synthesis lineage of SPADE-based label-to-image generation, but its central claim is that 2D slice-wise conditioning is insufficient for complex scans unless anatomical continuity across the slice direction is explicitly encoded (Wu et al., 15 Aug 2025).

2. Architectural organization

AnatoMaskGAN is SPADE-based and comprises an encoder, a 3D Spatial Noise Injection module, a GNN-based Slice Feature Fusion module, a SPADE-modulated decoder, multi-scale structural discriminators, and a Grayscale-Texture Classifier (Wu et al., 15 Aug 2025). The three components identified as the core innovations are the GNN-driven slice fusion module, the 3D spatial noise-injection strategy, and the grayscale-texture classifier.

Component	Role	Characterization
GNN-SIF	Inter-slice fusion	Slices as graph nodes with adjacency-guided attention
3D-SNI	Diversity injection	3D Gaussian noise aligned to physical volume
G-TC	Intensity-texture constraint	Class-wise grayscale and Sobel-texture feedback

The encoder extracts high-dimensional semantic features from each mask slice. The 3D-SNI module injects spatially aligned 3D Gaussian noise. The GNN-SIF module treats slices as graph nodes and fuses their features according to spatial adjacency and semantics. The SPADE-modulated decoder then converts the fused features into grayscale output slices (Wu et al., 15 Aug 2025). The discriminator side contains multi-scale structural discriminators for adversarial training and the auxiliary grayscale-texture classifier for class-wise texture and intensity supervision.

Semantic masks serve two distinct functions. First, they modulate normalization in the SPADE generator, providing region-wise semantic control. Second, they define one-hot regions $\mathrm{mask}_i$ for the grayscale-texture classifier, which computes per-class texture and grayscale statistics (Wu et al., 15 Aug 2025). The result is not merely mask-conditioned synthesis in the narrow pix2pix sense; it is mask-conditioned synthesis coupled to explicit cross-slice reasoning and class-conditioned texture control.

3. Inter-slice modeling, noise injection, and grayscale-texture control

The GNN-based strongly interdependent slice feature fusion module models spatial relationships between slices. Each slice $X_i$ is a node in a graph, a global node is added to aggregate holistic context, and edges encode spatial proximity so that directly adjacent slices are strongly connected while distant slices are weakly or uncorrelated (Wu et al., 15 Aug 2025). The adjacency matrix $\tilde{\mathbf A} \in \mathbb{R}^{N \times N}$ is built from slice indices and spatial distance, then vectorized as $$\mathrm{vec}(\tilde{\mathbf A}) \in \mathbb{R}^{N^2}$$.

Before graph fusion, encoder and decoder features are merged and enriched by multi-scale dilated convolutions. The fused feature $F_{\mathrm{fusion}}$ is passed through dilated convolutions at multiple rates, and the resulting representation is compressed back to $$F_{\mathrm{mfa}}\in\mathbb{R}^{C \times H \times W}$$ (Wu et al., 15 Aug 2025). The graph module then modulates these per-slice features through adjacency-conditioned attention:

$\begin{split} x'_{i,j,k} = x_{i,j,k} \, \sigma\Bigl( &\sum_{n=1}^{N_{1}} W_{(i,j,k),n}\, \delta\Bigl( \sum_{m=1}^{N^{2}} U_{n,m}\, \mathrm{vec}(\tilde{\mathbf A})_{m} + b_{n} \Bigr) \ &\; + c_{i,j,k} \Bigr) \end{split} \tag{1}$

This attention field is defined over $C \times H \times W$ and re-weights slice features according to graph-encoded inter-slice relationships, reinforcing neighbor-consistent features and suppressing inconsistent ones (Wu et al., 15 Aug 2025). In the paper’s interpretation, this supports continuous organ shapes, boundaries, and textures along the slice direction.

The 3D spatial noise-injection strategy introduces diversity without sacrificing continuity. A 3D noise volume $\{\mathrm{Slice}_i^G\}_{i=1}^{n}$0 is aligned with physical coordinates in the scan, partitioned along the $\{\mathrm{Slice}_i^G\}_{i=1}^{n}$1-axis, and injected through residual modulation:

$\{\mathrm{Slice}_i^G\}_{i=1}^{n}$2

Because $\{\mathrm{Slice}_i^G\}_{i=1}^{n}$3 is spatially continuous in 3D, adjacent slices receive correlated stochastic perturbations rather than independent slice-wise noise, which the paper describes as reducing “frame-jump” artifacts in 3D stacks (Wu et al., 15 Aug 2025).

The grayscale-texture joint classifier adds a complementary constraint not handled by standard adversarial or VGG perceptual losses. Its texture branch uses Sobel gradients and its grayscale branch uses masked region statistics—mean, standard deviation, maximum, and minimum—computed per anatomical class (Wu et al., 15 Aug 2025). The resulting GrayTexture loss penalizes deviations between generated and real class-wise grayscale-texture scores, targeting the “plastic” appearance that can survive even when global anatomy is plausible.

4. Objective function and optimization

The generator objective combines adversarial, feature-matching, perceptual, and grayscale-texture terms:

$\{\mathrm{Slice}_i^G\}_{i=1}^{n}$4

with $\{\mathrm{Slice}_i^G\}_{i=1}^{n}$5, $\{\mathrm{Slice}_i^G\}_{i=1}^{n}$6, and $\{\mathrm{Slice}_i^G\}_{i=1}^{n}$7 (Wu et al., 15 Aug 2025). The discriminator term is the standard conditional GAN loss; feature matching is computed across discriminator layers; and the VGG term is an $\{\mathrm{Slice}_i^G\}_{i=1}^{n}$8 perceptual loss on deep features. No explicit GNN-specific regularization term is introduced, since GNN-SIF is trained end-to-end through the generator losses (Wu et al., 15 Aug 2025).

Training uses PyTorch on an NVIDIA A800 GPU with Adam, $\{\mathrm{Slice}_i^G\}_{i=1}^{n}$9, $\mathrm{mask}_i$0, and learning rate $\mathrm{mask}_i$1 (Wu et al., 15 Aug 2025). The paper reports 100 training iterations, resize to $\mathrm{mask}_i$2, normalization to $\mathrm{mask}_i$3, and a 70/15/15 training-validation-test split, together with slice-level rotations and flips as augmentation (Wu et al., 15 Aug 2025). The backbone is SPADE throughout.

A notable aspect of the optimization design is that structural realism and radiologically meaningful grayscale-texture statistics are treated as separate supervisory channels. This suggests a division of labor in which the discriminators police global structural realism, the VGG term stabilizes perceptual correspondence, and G-TC regularizes anatomy-conditioned tonal and gradient distributions.

5. Datasets, metrics, and empirical performance

Evaluation uses PSNR, SSIM, and LPIPS, with higher PSNR and SSIM and lower LPIPS interpreted as better reconstruction fidelity and perceptual quality (Wu et al., 15 Aug 2025). On L2R-OASIS, AnatoMaskGAN is reported as G3-Net in the comparison table and attains PSNR 26.50 dB, SSIM 0.9229, and LPIPS 0.0559; the strongest listed baseline, OASIS, attains PSNR 26.07, SSIM 0.9044, and LPIPS 0.0922 (Wu et al., 15 Aug 2025). On L2R-Abdomen CT, AnatoMaskGAN attains PSNR 21.98 dB, SSIM 0.8602, and LPIPS 0.0807; the paper highlights the SSIM value as a 0.48 percentage-point gain over the best baseline (Wu et al., 15 Aug 2025).

Dataset	Best baseline reported	AnatoMaskGAN
L2R-OASIS	OASIS: 26.07 / 0.9044 / 0.0922	26.50 / 0.9229 / 0.0559
L2R-Abdomen CT	SAFM: 22.35 / 0.8386 / 0.0896; OASIS: 20.45 / 0.8554 / 0.0991	21.98 / 0.8602 / 0.0807

The ablation study isolates the three principal modules. Starting from SPADE, adding GNN slice fusion raises OASIS performance from PSNR 24.50, SSIM 0.8871, LPIPS 0.1036 to PSNR 26.22, SSIM 0.9115, LPIPS 0.0570 (Wu et al., 15 Aug 2025). Adding 3D noise yields PSNR 26.42, SSIM 0.9116, LPIPS 0.0567, and the full model with G-TC reaches PSNR 26.50, SSIM 0.9229, LPIPS 0.0559 (Wu et al., 15 Aug 2025). On Abdomen CT, the trajectory is less monotone at the intermediate stage—SSIM fluctuates slightly when 3D noise is introduced—but the full model still provides the best PSNR, SSIM, and LPIPS (Wu et al., 15 Aug 2025).

Qualitatively, the paper attributes sharper cortical folds, clearer gray/white matter boundaries, and reduced artifacts in brain MRI to the combined effect of graph fusion and grayscale-texture control, while in abdominal CT it emphasizes crisper organ edges, intact shapes, and more realistic CT texture (Wu et al., 15 Aug 2025). The broad empirical pattern is therefore not only higher fidelity in a pixel or structural sense, but also more stable slice-to-slice coherence.

6. Relation to earlier and adjacent mask-guided generative models

Although the exact title AnatoMaskGAN belongs to the 2025 multi-slice medical synthesis framework, several earlier papers established neighboring ideas under other names. In portraits, "Mask-Guided Portrait Editing with Conditional GANs" treated the face as a set of semantic components with separate feature embeddings and a target semantic mask controlling geometry and layout, thereby disentangling appearance from structure for face synthesis, Face Swap+, and local manipulation (Gu et al., 2019). That work is anatomy-aware in a facial-part sense, but it does not use the name AnatoMaskGAN.

In lesion synthesis, "Mask2Lesion: Mask-Constrained Adversarial Skin Lesion Image Synthesis" used binary lesion masks as structural priors so that the synthesized dermoscopic image was strictly confined to the mask, and the input mask served directly as the segmentation label for downstream augmentation (Abhishek et al., 2019). The central idea there is not inter-slice modeling but label consistency by construction.

In unpaired cross-modality synthesis, "Structure-Preserving Synthesis: MaskGAN for Unpaired MR-CT Translation" used automatically extracted coarse masks, a two-branch mask/content generator, mask supervision, and cycle shape consistency to preserve anatomy without expert segmentation labels on a heavily misaligned pediatric MR-CT dataset (Phan et al., 2023). Its contribution is structural consistency under unpaired translation rather than semantic-mask-conditioned label-to-image synthesis.

A later unsupervised anomaly-detection work, "Latent Anomaly Detection: Masked VQ-GAN for Unsupervised Segmentation in Medical CBCT," explicitly interprets its masked VQ-GAN as an anatomy-aware masked generative framework: it trains on healthy jaw anatomy, applies anatomy-specific teeth and mandibular-canal masking, and uses differences between a baseline reconstruction and a healthy-inpainting reconstruction for anomaly localization (Wang, 17 Jun 2025). This suggests that the name AnatoMaskGAN has acquired a broader descriptive use for anatomy-aware masked generative modeling, even when the concrete architecture differs substantially from SPADE plus GNN slice fusion.

Taken together, these papers mark a progression from mask-conditioned geometric control, to mask-preserving augmentation, to annotation-free structure-preserving translation, and finally to explicit multi-slice graph reasoning with anatomy-conditioned texture control. A plausible implication is that AnatoMaskGAN is best understood both as a specific 2025 medical synthesis model and as part of a broader family of mask-guided, anatomy-aware generative systems.

7. Limitations and future directions

The AnatoMaskGAN paper identifies three limitations: cross-modal stability remains non-trivial for modalities such as PET and ultrasound, GNN-SIF introduces computational overhead, and performance may still be limited when data are extremely scarce (Wu et al., 15 Aug 2025). The proposed future directions are more efficient slice-relationship encoders, including lightweight graph or transformer variants, cross-modal alignment and contrast generation, and few-shot or self-supervised pre-training for rare-disease and privacy-restricted settings (Wu et al., 15 Aug 2025).

Within the broader mask-guided literature, a recurring issue is dependence on the quality and granularity of the structural prior. Portrait systems depend on accurate parsing masks and relatively coarse semantic components (Gu et al., 2019); lesion synthesis depends on clinically meaningful mask generation rather than arbitrary shapes (Abhishek et al., 2019); MR-CT translation with coarse masks depends on stable extraction of foreground structure under domain shift (Phan et al., 2023). This suggests that gains from anatomy-aware mask guidance are tightly coupled to the informativeness of the conditioning signal itself.

For medical semantic synthesis specifically, the main conceptual contribution of AnatoMaskGAN is the claim that semantic masks should not be treated as isolated 2D conditioning maps when the target anatomy is volumetric. In that view, inter-slice adjacency, spatially coherent stochasticity, and class-wise grayscale-texture statistics are not auxiliary refinements but integral components of anatomically consistent generation (Wu et al., 15 Aug 2025).