---
title: 'GANeXt: 3D Patch-Based GAN for CT Synthesis'
url: https://www.emergentmind.com/topics/ganext
type: topic
---

# GANeXt: 3D Patch-Based GAN for CT Synthesis

GANeXt is a 3D, patch-based generative adversarial network (GAN) framework designed for the synthesis of computed tomography (CT) volumes from both magnetic resonance imaging (MRI) and cone-beam CT (CBCT), supporting unified, multi-region sCT generation crucial for adaptive radiotherapy. GANeXt features a fully convolutional U-shaped generator built entirely from 3D ConvNeXt blocks and utilizes a conditional PatchGAN discriminator, with multi-head segmentation for CBCT-to-CT tasks. The architecture harnesses a suite of loss functions—including voxelwise, perceptual, segmentation-aware, and adversarial losses—and employs robust training, augmentation, and inference protocols to maximize synthesis fidelity and anatomical quality [2512.19336].

## 1. Architectural Overview

GANeXt’s architecture revolves around two principal modules: a 3D U-shaped generator based on ConvNeXt blocks (denoted “GeNeXt” in this context), and a conditional PatchGAN discriminator.

### ConvNeXt-Enhanced Generator

- **GeNeXt Block:** Each block receives an input feature map $X\in\mathbb{R}^{C\times D\times H\times W}$, processes it by applying a $3\times3\times3$ depthwise convolution (DWConv), instance normalization, GELU activation, channel expansion via pointwise convFFN, and a residual connection. Expansion ratios $R_i$ are stage-specific.
- **Downsampling/Up-sampling:** Downsampling blocks apply stride-2 DWConvs; shortcut paths use $1\times1\times1$ “resize-convs.” Upsampling blocks use stride-2 transposed DWConvs.
- **U-Shape Composition:** Five stages are organized as encoder (four downsampling), bottleneck, and decoder (four upsampling) with skip connections at each decoder stage, realized by channel-wise concatenation and $1\times1\times1$ projection. Channel progression is $\{32, 64, 128, 256, 512\}$; blocks per stage: $\{3,4,6,6,6\}$.
- **Patch-based Processing:** During training, 3D patches (MRI-to-CT: $1\times32\times160\times192$; CBCT-to-CT: $1\times32\times128\times128$) are extracted for generator input.

### Conditional PatchGAN Discriminator

- **Design:** Receives conditioning/generation pairs along the channel dimension, processed by cascaded $4\times4\times4$ strided convolutions (LeakyReLU, instance norm), downsampling to a receptive field of $16^3$ voxels.
- **Outputs:** Main head gives a $1\times D/16\times H/16\times W/16$ map of real/fake logits; for CBCT-to-CT synthesis, an additional multi-class segmentation head applies a shallow decoder to re-upsample and predict anatomical structures.

## 2. Loss Functions

GANeXt aggregates multiple complementary objectives during training:

| Loss Term                         | Symbol                  | MRI→CT Weight | CBCT→CT Weight |
|------------------------------------|-------------------------|---------------|---------------|
| Mean Absolute Error                | $\mathcal{L}_{\mathrm{mae}}$  | $10$          | $10$          |
| Perceptual Loss (ConvNeXt-B LPIPS) | $\mathcal{L}_{\mathrm{perc}}$ | $1$           | $1$           |
| Segmentation-Masked MAE            | $\mathcal{L}_{\mathrm{mask}}$ | $50$          | $10$          |
| Adversarial Loss                   | $\mathcal{L}_{\mathrm{adv}}$  | $10$          | $10$          |
| Feature-Matching Loss              | $\mathcal{L}_{\mathrm{fm}}$   | $10$          | $10$          |
| Segmentation Loss (only CBCT→CT)   | $\mathcal{L}_{\mathrm{seg}_g}$| $0$           | $0.5$         |

- **Mean Absolute Error:** Promotes voxelwise intensity fidelity.
- **Perceptual Loss:** Utilizes ConvNeXt-B LPIPS to preserve structural realism at feature levels.
- **Segmentation-Masked MAE:** Restricts error assessment to anatomically relevant regions using a pretrained TotalSegmentator mask.
- **Adversarial Loss:** Implements standard cGAN formulation.
- **Feature-Matching:** Encourages generator alignment with discriminator features.
- **Segmentation Loss:** Applies Dice and cross-entropy (multi-class) to enforce anatomical accuracy during CBCT-to-CT synthesis; separate components for discriminator ($\mathcal{L}_{\mathrm{seg}_d}$) and generator ($\mathcal{L}_{\mathrm{seg}_g}$).

### Generator Objectives

- **MRI-to-CT:** $\mathcal{L}_{\mathrm{G}} = 10\,\mathcal{L}_{\mathrm{mae}} + 1\,\mathcal{L}_{\mathrm{perc}} + 50\,\mathcal{L}_{\mathrm{mask}} + 10\,\mathcal{L}_{\mathrm{adv}} + 10\,\mathcal{L}_{\mathrm{fm}}$
- **CBCT-to-CT:** Identical, with segmentation loss $\mathcal{L}_{\mathrm{seg}_g}$ added at $0.5$ weight.

### Discriminator Objective (CBCT-to-CT)

$\mathcal{L}_{\mathrm{D}} = - \mathbb{E}\bigl[\log D(x,y)\bigr] - \mathbb{E}\bigl[\log(1-D(x,G(x)))\bigr] + 0.5\,\mathcal{L}_{\mathrm{seg}_d}$

## 3. Training Protocol

The training leverages modern optimization and regularization components:

- **Optimizers:** Generator: AdamW ($\text{lr}=5\times10^{-4}$, weight decay $0.01$); Discriminator: AdamW for MRI-to-CT, Adam for CBCT-to-CT ($\text{lr}=1\times10^{-3}$).
- **Schedulers:** Linear warmup for first 5% epochs, followed by cosine annealing to zero (MRI-to-CT: 3000 epochs; CBCT-to-CT: 1000 epochs).
- **Batch Size:** 8 per iteration, distributed over 4 NVIDIA A100 GPUs.
- **Model Selection:** Final checkpoint is used at epoch cutoff without further fine-tuning.
- **Algorithmic Workflow:** Each batch undergoes augmentation, generator and discriminator both updated separately, and learning-rate schedulers progressed per batch. Explicit pseudocode structures this process.

## 4. Data Preprocessing and Augmentation

GANeXt applies rigorous preprocessing steps:

- **Deformable Registration:** All CTs are deformably registered to the challenge test-set reference for spatial correspondence with MRI/CBCT.
- **Foreground Cropping:** Anatomical region isolation via simple body mask generation (thresholding plus morphology).
- **Intensity Normalization:**
  - Inputs (MRI/CBCT): Percentile clipping ([0.5th, 99.5th]) and scaling to $[0,1]$.
  - CT targets: Linear mapping of Hounsfield Units to $[0,1]$ within modality-specific ranges: $[-1024,1000]$ for MRI-to-CT, $[-1024,1500]$ for CBCT-to-CT.
- **Augmentation:**
  - **MRI-to-CT Only:** Random zooming (scale in $[0.8,1.3]$).
  - Both tasks: Randomly cropped to fixed patch sizes, random horizontal/vertical flipping.

## 5. Inference and Output Reconstruction

Inference employs an efficient sliding-window strategy:

- **Sliding-Window Inference:** Overlap of $0.8$ in each dimension; each patch is processed and the predictions are reconstructed by voxelwise averaging (“average folding”) to maintain consistency across overlapping regions.
- **Postprocessing:** CT normalization is inverted to restore Hounsfield units:
  $$
  \mathrm{HU} = \mathrm{clip}(\hat y, 0, 1)\cdot(\mathrm{maxHU}-\mathrm{minHU}) + \mathrm{minHU}.
  $$
- **No Fine-Tuning:** After joint training across all anatomical regions, the selected models undergo no additional region- or modality-specific adaptation.

## 6. Context and Performance

GANeXt’s stacking of 3D ConvNeXt modules within a U-Net topological framework, combined with multi-term regularization and adversarial objectives, yields state-of-the-art synthetic CT outputs for both MRI-to-CT and CBCT-to-CT targets. The approach remains purely convolutional and computationally efficient, while accommodating segmentation-based anatomical guidance for further accuracy in challenging CBCT-to-CT translation [2512.19336].

A plausible implication is that GANeXt’s architectural and loss-function innovations facilitate generalizable, high-fidelity medical image synthesis across multiple modalities and anatomical domains without recourse to modality- or region-specific retraining or post hoc adaptation.

Source: https://www.emergentmind.com/topics/ganext