SynBT: Diffusion-based Breast Tumor Synthesis

• SynBT is a 3D diffusion model that generates diverse, anatomically plausible breast tumor appearances in contrast-enhanced MRI.
• It combines a patch-to-volume VQ-VAE for efficient latent encoding with a mask-conditioned latent diffusion model for controllable tumor synthesis.
• Empirical evaluation shows SynBT improves segmentation performance by up to 3 Dice points over traditional volume-based methods.

SynBT is a 3D diffusion-based framework for synthesizing breast tumors in contrast-enhanced MRI, developed to improve downstream breast tumor segmentation by expanding the diversity of tumor appearances, sizes, and locations seen during training. Its central premise is that breast DCE-MRI poses a large-field-of-view volumetric synthesis problem for which prior patch-based tumor generators are a poor fit: tumors may occupy large spatial volumes, and anatomical and enhancement context outside a small local patch materially affects both realism and segmentation utility. SynBT addresses this by combining a patch-to-volume VQ-VAE with a mask-conditioned latent diffusion model that synthesizes tumor appearance inside selected breast tissue regions, and then using the resulting images as augmentation for 3D segmentation networks (Yang et al., 3 Sep 2025).

1. Clinical setting and methodological motivation

SynBT is situated in the problem of breast tumor segmentation in high-resolution, large-field-of-view 3D contrast-enhanced MRI. The motivating observation is that synthetic tumors can provide controllable variability for segmentation training when annotated tumor data are limited, expensive to collect, and may not span the full range of clinically relevant morphologies, enhancement patterns, acquisition conditions, and patient anatomy. In this setting, the goal of synthesis is not image generation as an end in itself, but segmentation improvement through more diverse and anatomically plausible lesion augmentation (Yang et al., 3 Sep 2025).

The method is explicitly designed against the limitations of earlier tumor synthesis approaches that operate on small local patches, commonly around $96^3$ voxels or smaller. Such methods are most suitable when local context is sufficient, but breast DCE-MRI requires broader spatial context because tumors exist inside a much larger breast volume. The paper argues that patch-based generation is insufficient when tumors occupy large spatial volumes or when global breast context is necessary for realism and segmentation performance. It further reports that small patches fail to capture enough semantic information from the breast region and that patch-only reconstruction can introduce grid effects.

This large-FOV emphasis is also reflected in the downstream segmentation setting. SynBT is intended for segmentation models that operate on larger 3D fields of view rather than lesion-centered microcrops. A plausible implication is that the method’s value depends as much on preserving volumetric coherence as on producing locally realistic tumor texture.

2. Patch-to-volume latent compression

The first stage of SynBT is a Patch-to-Volume VQ-VAE that compresses high-resolution dual-channel MRI volumes into a compact latent representation while preserving the spatial arrangement of the whole breast. The input is a two-channel DCE-MRI volume obtained by concatenating a pre-contrast MRI and one post-contrast MRI:

$$\mathbf{X} = \mathrm{Concat}(\mathbf{X}_{pre}, \mathbf{X}_{post}) \in \mathbb{R}^{H \times W \times D \times 2}.$$

Instead of encoding the full volume directly, the image is divided into 3D patches,

$$\mathbf{x}_i \in \mathbb{R}^{h \times w \times d \times 2},$$

and each patch is mapped by an encoder $E(\cdot)$ to a latent code,

$\mathbf{z}_i = E(\mathbf{x}_i).$

The patch-level latent vectors are then reassembled according to their original spatial positions into a latent volume $\mathbf{Z}$. Quantization is performed with a VQ codebook

$\mathcal{C} = \{\mathbf{e}_k\}_{k=1}^K,$

and the decoder reconstructs the image volume from the quantized latent volume:

$\hat{\mathbf{X}} = D(\hat{\mathbf{Z}}).$

Training proceeds in two steps. In the first step, the VQ-VAE is trained on random sub-patches to learn local representations efficiently. In the second step, the model is adapted for volume-level reconstruction by tiling the input volume into patches, aggregating the latent patches back into a full latent volume, and reconstructing at volume level. During this second step, the encoder and codebook are frozen and only the decoder is fine-tuned. The paper states that this reduces grid artifacts across patch boundaries and allows the encoder to focus on regional information while the decoder learns globally coherent reconstruction (Yang et al., 3 Sep 2025).

The reported implementation is concrete. The downstream MRI crop size is $128 \times 192 \times 192 \times 2$. The VQ-VAE patch size is $64^3 \times 2$, with compression rate 4. The codebook contains 8192 codes of dimension 8. The initial convolutional channel count is 32, doubling as patch resolution decreases. Patch-level VQ-VAE training uses batch size 8 and learning rate $$\mathbf{X} = \mathrm{Concat}(\mathbf{X}_{pre}, \mathbf{X}_{post}) \in \mathbb{R}^{H \times W \times D \times 2}.$$0; volume-level fine-tuning uses batch size 1 and learning rate $$\mathbf{X} = \mathrm{Concat}(\mathbf{X}_{pre}, \mathbf{X}_{post}) \in \mathbb{R}^{H \times W \times D \times 2}.$$1 for 100,000 steps. A discriminator is introduced after 200,000 steps, and the adversarial and perceptual loss weights are 0.01 and 0.001, respectively. The exact layer-by-layer encoder-decoder architecture and the explicit VQ commitment/codebook loss are not specified.

3. Mask-conditioned latent diffusion and anatomically informed placement

Once the latent representation is available, SynBT trains a mask-conditioned latent diffusion model to synthesize tumor appearance. The model does not regenerate the entire breast anatomy from scratch. Instead, it removes or masks the tumor from a tumor-present DCE-MRI volume, encodes the masked image to obtain a background latent volume, and conditions diffusion on both that latent background and a downsampled tumor mask:

$$\mathbf{X} = \mathrm{Concat}(\mathbf{X}_{pre}, \mathbf{X}_{post}) \in \mathbb{R}^{H \times W \times D \times 2}.$$2

This formulation means that the model is conditioned simultaneously on tumor location and surrounding breast context. The generation target is therefore tumor appearance to be inpainted into the masked region in latent space rather than unconditional whole-volume synthesis (Yang et al., 3 Sep 2025).

The diffusion process follows the standard DDPM formulation. The forward process is

$$\mathbf{X} = \mathrm{Concat}(\mathbf{X}_{pre}, \mathbf{X}_{post}) \in \mathbb{R}^{H \times W \times D \times 2}.$$3

and the reverse process is learned by a neural network $$\mathbf{X} = \mathrm{Concat}(\mathbf{X}_{pre}, \mathbf{X}_{post}) \in \mathbb{R}^{H \times W \times D \times 2}.$$4 conditioned on the masked-background latent and tumor mask:

$$\mathbf{X} = \mathrm{Concat}(\mathbf{X}_{pre}, \mathbf{X}_{post}) \in \mathbb{R}^{H \times W \times D \times 2}.$$5

with

$$\mathbf{X} = \mathrm{Concat}(\mathbf{X}_{pre}, \mathbf{X}_{post}) \in \mathbb{R}^{H \times W \times D \times 2}.$$6

The denoised latent volume is then passed through the VQ-VAE codebook and decoder to reconstruct the final high-resolution MRI with synthetic tumor.

Tumor placement is governed by an explicit controllability pipeline. Binary tumor masks come from a bank of real tumor templates extracted from the training data and can be augmented by flipping, rotation, zoom, and similar transforms. Placement is constrained to fibroglandular breast tissue, because that is where realistic placement is expected. To support this, the authors train a tissue segmentation model using fibroglandular masks from the Duke dataset via a SynthSeg-style pipeline. The rest of the body is clustered by k-means with a random number of clusters between 2 and 8, and the tissue masks are post-processed to select likely regions for synthetic tumor placement. This makes SynBT’s conditioning anatomically informed rather than a free-form binary mask inserted arbitrarily.

4. Training protocol, data, and downstream segmentation

The diffusion backbone is described only as a “specially designed UNet” used for latent generation; the paper does not specify the number of resolution levels, residual blocks, attention blocks, normalization layers, or channel multipliers. Diffusion training runs for 500,000 iterations with learning rate $$\mathbf{X} = \mathrm{Concat}(\mathbf{X}_{pre}, \mathbf{X}_{post}) \in \mathbb{R}^{H \times W \times D \times 2}.$$7. The schedule uses 1000 timesteps, and inference is accelerated to 10 sampling steps with DDIM (Yang et al., 3 Sep 2025).

Synthetic tumor generation is used as on-the-fly augmentation for downstream segmentation. The segmentation models are trained jointly on real tumor images and synthetically generated tumor images. The crop size is again $$\mathbf{X} = \mathrm{Concat}(\mathbf{X}_{pre}, \mathbf{X}_{post}) \in \mathbb{R}^{H \times W \times D \times 2}.$$8, corresponding to latent size $$\mathbf{X} = \mathrm{Concat}(\mathbf{X}_{pre}, \mathbf{X}_{post}) \in \mathbb{R}^{H \times W \times D \times 2}.$$9. Three 3D segmentation backbones are evaluated through MONAI implementations: U-Net, nnU-Net, and SwinUNETR. They are trained for 300 epochs with learning rate $$\mathbf{x}_i \in \mathbb{R}^{h \times w \times d \times 2},$$0, batch size 1, and standard data augmentations including flipping, rotation, intensity scaling, and shifting. Full-volume inference uses a sliding-window strategy, and the best validation checkpoint is selected. The exact segmentation loss is not reported.

Evaluation uses the public MAMA-MIA breast DCE-MRI benchmark dataset. The official split provides 1200 images for training; the remaining 306 images are divided into 100 for validation and 206 for testing. All images are isotropically resampled to $$\mathbf{x}_i \in \mathbb{R}^{h \times w \times d \times 2},$$1 mm. Because multiple post-contrast sequences are available, a random post-contrast image is selected during training to pair with the pre-contrast image, whereas the first post-contrast sequence is used in validation and testing. Tumor masks from the training set are harvested to build the tumor-template bank. Input volumes are randomly cropped to $$\mathbf{x}_i \in \mathbb{R}^{h \times w \times d \times 2},$$2. The reported evaluation metrics are Dice Score (DSC), 95% Hausdorff Distance (95HD), and mean surface distance (MSD), computed via MedPy.

5. Empirical performance and ablation evidence

The central empirical claim is that SynBT improves segmentation performance by roughly 2–3 Dice points, with the strongest alignment to that claim appearing in SwinUNETR. The experiments compare four settings for each backbone: small-patch segmentation on $$\mathbf{x}_i \in \mathbb{R}^{h \times w \times d \times 2},$$3 crops, volume-based segmentation on large crops, synthetic augmentation with SynTumor, and synthetic augmentation with SynBT (Yang et al., 3 Sep 2025).

U-Net

• patch-UNet: DSC $$\mathbf{x}_i \in \mathbb{R}^{h \times w \times d \times 2},$$4, 95HD $$\mathbf{x}_i \in \mathbb{R}^{h \times w \times d \times 2},$$5 mm, MSD $$\mathbf{x}_i \in \mathbb{R}^{h \times w \times d \times 2},$$6 mm
• Volume-UNet: DSC $$\mathbf{x}_i \in \mathbb{R}^{h \times w \times d \times 2},$$7, 95HD $$\mathbf{x}_i \in \mathbb{R}^{h \times w \times d \times 2},$$8 mm, MSD $$\mathbf{x}_i \in \mathbb{R}^{h \times w \times d \times 2},$$9 mm
• SynTumor-UNet: DSC $E(\cdot)$0, 95HD $E(\cdot)$1 mm, MSD $E(\cdot)$2 mm
• SynBT-UNet: DSC $E(\cdot)$3, 95HD $E(\cdot)$4 mm, MSD $E(\cdot)$5 mm

nnU-Net

• patch-nnUNet: DSC $E(\cdot)$6, 95HD $E(\cdot)$7 mm, MSD $E(\cdot)$8 mm
• Volume-nnUNet: DSC $E(\cdot)$9, 95HD $\mathbf{z}_i = E(\mathbf{x}_i).$0 mm, MSD $\mathbf{z}_i = E(\mathbf{x}_i).$1 mm
• SynTumor-nnUNet: DSC $\mathbf{z}_i = E(\mathbf{x}_i).$2, 95HD $\mathbf{z}_i = E(\mathbf{x}_i).$3 mm, MSD $\mathbf{z}_i = E(\mathbf{x}_i).$4 mm
• SynBT-nnUNet: DSC $\mathbf{z}_i = E(\mathbf{x}_i).$5, 95HD $\mathbf{z}_i = E(\mathbf{x}_i).$6 mm, MSD $\mathbf{z}_i = E(\mathbf{x}_i).$7 mm

SwinUNETR

• patch-SwinUNETR: DSC $\mathbf{z}_i = E(\mathbf{x}_i).$8, 95HD $\mathbf{z}_i = E(\mathbf{x}_i).$9 mm, MSD $\mathbf{Z}$0 mm
• Volume-SwinUNETR: DSC $\mathbf{Z}$1, 95HD $\mathbf{Z}$2 mm, MSD $\mathbf{Z}$3 mm
• SynTumor-SwinUNETR: DSC $\mathbf{Z}$4, 95HD $\mathbf{Z}$5 mm, MSD $\mathbf{Z}$6 mm
• SynBT-SwinUNETR: DSC $\mathbf{Z}$7, 95HD $\mathbf{Z}$8 mm, MSD $\mathbf{Z}$9 mm

Relative to the large-volume baseline, SynBT improves Dice by about 1.1 points for U-Net, 1.66 points for nnU-Net, and about 3.0 points for SwinUNETR. Across all three architectures, SynBT is reported as the best method on DSC, 95HD, and MSD. The patch-based segmentation variants perform dramatically worse than the volume-based variants, which functions as an ablation on the importance of large field of view. The comparison with SynTumor is also informative: on this task, the analytical texture-generation baseline is consistently worse than both the volume baseline and SynBT. Qualitatively, the paper reports that SynBT produces more realistic tumors than SynTumor, with more plausible enhancement patterns and fewer visibly artificial textures. It also reports that patch-only VQ-VAE reconstruction exhibits visible grid artifacts, whereas the Patch-to-Volume VQ-VAE reduces these artifacts and better preserves realism.

6. Limitations, significance, and nomenclature

The reported strengths of SynBT are tightly connected to its problem formulation. It is presented as one of the first methods in this space to demonstrate high-quality synthetic breast tumor generation in large-scale public DCE-MRI and to link that generation directly to measurable segmentation gains. By generating only the tumor foreground rather than the whole anatomy, the method makes the synthesis problem more tractable and more data-efficient. Its use of real tumor masks together with tissue-informed placement provides controllability over position, size, and shape, while latent diffusion is used to improve texture realism relative to analytical synthesis baselines (Yang et al., 3 Sep 2025).

The paper also identifies several limitations. Tumor placement still assumes that tumors occur in random fibroglandular tissue locations; more precise clinical modeling of likely tumor positions could improve realism. The tissue-segmentation pipeline used to identify candidate placement regions requires substantial post-processing effort. The diffusion model does not provide fine-grained control over tumor texture semantics, such as explicit control over subtypes or categories. Dedicated quantitative synthesis-quality analyses such as FID, SSIM, or radiomics similarity were not performed. More broadly, the method depends on enough real tumor examples to build a meaningful tumor-mask bank and to train the generative model; it is augmentation rather than replacement for real data. The full workflow is also computationally substantial, requiring VQ-VAE training, diffusion training, and segmentation training on a DGX A100 GPU.

A common source of confusion is nomenclature. In the literature supplied here, the exact term “SynBT” refers to the breast tumor synthesis framework just described. Related abbreviations are used in unrelated domains: BitTorrent Sync has been discussed as a peer-to-peer synchronization system relevant to “SynBT/SyncBT” naming, but that usage concerns forensic network analysis rather than medical image synthesis (Scanlon et al., 2015); and in DSMC literature, “SynBT” is not an official algorithm name, with the relevant terms instead being symmetrized Bernoulli-trials variants such as SSBT and SGBT (Shoja-sani et al., 8 Jun 2025). In the breast MRI literature, however, SynBT specifically denotes the 3D latent diffusion framework for controllable breast tumor synthesis and segmentation augmentation.