---
title: Generalised Dice Loss for Segmentation
url: https://www.emergentmind.com/topics/generalised-dice-loss-gdl
type: topic
---

# Generalised Dice Loss for Segmentation

Generalised Dice Loss (GDL) is a deep learning loss function designed to address severe class imbalance in multi-class image segmentation, particularly in medical images where rare structures are often encountered. GDL extends the classical Dice coefficient by introducing dynamic, per-class rebalancing through inverse squared-volume weighting, thereby ensuring robust optimization and balanced gradient contributions across classes, regardless of their frequency or size [1707.03237].

## 1. Mathematical Formulation

The Generalised Dice Loss is defined for an input patch or image with $L$ classes (including background) and $N$ voxels (or pixels). For each class $l$ and voxel $n$:

- $r_{ln} \in \{0,1\}$ is the one-hot ground-truth label for voxel $n$ in class $l$
- $p_{ln} \in [0,1]$ is the predicted probability for voxel $n$ and class $l$
- $w_l \geq 0$ is the class-specific weight

The GDL is given by:
$$
\mathrm{GDL} = 1 - 2 \cdot \frac{\sum_{l=1}^L w_l \sum_{n=1}^N r_{ln} p_{ln}}{\sum_{l=1}^L w_l \left( \sum_{n=1}^N r_{ln} + \sum_{n=1}^N p_{ln} \right)}
$$

In compact notation:
$$
\mathrm{GDL} = 1 - 2\frac{\sum_{l} w_l \langle r_l, p_l \rangle}{\sum_{l} w_l (|r_l| + |p_l|)}
$$
where $\langle r_l, p_l \rangle = \sum_n r_{ln}p_{ln}$ and $|r_l| = \sum_n r_{ln}$.

## 2. Class-Rebalancing and Weighting Scheme

GDL introduces a class rebalancing mechanism by assigning each class a weight inversely proportional to the squared volume of the ground-truth for that class:
$$
w_l = \frac{1}{\left( \sum_{n=1}^N r_{ln} \right)^2 + \epsilon}
$$
where $\epsilon$ (typically $10^{-6}$) is used to prevent division by zero. This “inverse-volume²” approach ensures that small classes receive larger weights, so their segmentation error is amplified to match the influence of larger, more prevalent classes. As a result, the bias of standard Dice toward large regions is mitigated and per-class gradient magnitudes are balanced. This weighting is dynamic and recalculated per batch during training.

## 3. Theoretical Properties and Comparative Motivation

GDL is characterized by several theoretical and practical advantages over alternative loss functions such as weighted cross-entropy and standard Dice loss:

- **Class-balanced overlap:** GDL automatically ensures every class—including rare ones—contributes equally to the loss within each batch, adapting class weights according to their occurrence.
- **Robustness to imbalance:** Weighted cross-entropy may excessively upweight small classes or produce vanishing gradients. Sensitivity–specificity losses require manual trade-off parameter tuning. GDL, by contrast, achieves balanced optimization without manual class-specific tuning.
- **Scale invariance:** Because region sizes can vary greatly across samples, GDL ensures that the contribution of each class to the loss remains approximately constant.
- **Gradient stability:** Empirical analysis shows GDL gradients remain well-conditioned even for regions comprising less than 0.1% of voxels, a regime where other losses often become unstable or lead to slow convergence.

## 4. Empirical Evaluation

Experiments were conducted on both 2D and 3D segmentation tasks involving extreme class imbalance:

- **2D Segmentation:** The BRATS brain tumor segmentation dataset was used, where tumor pixels could constitute as little as 0.5% of a patch. Architectures tested included UNet and TwoPathCNN. GDL consistently yielded the highest (or near-highest) Dice Similarity Coefficient (DSC) across learning rates and patch sizes. For example, at learning rate $10^{-4}$ (small patch), UNet achieved DSCs: DL₂=0.84, SS=0.82, WCE=0.83, GDL=0.85.
- **3D Segmentation:** An in-house white-matter hyperintensity dataset with 524 subjects was evaluated using DeepMedic and HighResNet. Lesion volumes were often less than 0.02% of the patch. GDL performed best, and was uniquely robust to learning-rate selection and class imbalance (HighResNet: GDL=0.65 vs. DL₂=0.62, SS=0.58, WCE non-convergent at LR $10^{-4}$, large patch).

Test-set evaluation with HighResNet on the 3D WMH task yielded median DSCs of 0.66 for GDL (best), compared to 0.63 (DL₂), 0.60 (SS), with WCE failing to converge under these conditions.

## 5. Practical Implementation

GDL is implementable as a batch-wise loss computed over predicted probability maps (typically softmax outputs) and one-hot ground-truth labels. Efficient vectorized computation, as described below, is critical for workflow integration:

```python
def generalized_dice_loss(pred, target, epsilon=1e-6):
    # pred: (B, L, D, H, W) probabilities after softmax
    # target: (B, L, D, H, W) one‐hot ground truth
    w_l = 1.0 / ((target.sum(dim=(0,2,3,4))**2) + epsilon)  # shape: (L,)
    intersection = (pred * target).sum(dim=(0,2,3,4))        # (L,)
    cardinality = (pred + target).sum(dim=(0,2,3,4))         # (L,)
    num = 2 * (w_l * intersection).sum()
    den = (w_l * cardinality).sum() + epsilon
    loss = 1.0 - num / den
    return loss
```
Training protocols recommend:
- Optimizer: SGD or Adam, with a learning rate of $10^{-4}$ providing stable trade-off between speed and convergence
- Batch size: maximize within GPU constraints; ensure all classes are present at least once per batch to prevent weight blow-up
- Patch size: smaller patches and larger batch sizes yield more stable class-volume estimates and improved convergence

## 6. Limitations and Best Practices

- **Extreme imbalance instability:** When a class is absent from a batch (i.e., class volume approaches zero), its weight $w_l$ diverges. Mitigations include adding a numerical $\epsilon$, explicitly capping $w_l$, or enforcing sampling to ensure each class is present in every batch.
- **Increased memory overhead:** Because GDL requires global class sums per batch, small batch sizes may introduce estimation noise into the weights and loss.
- **Hybrid loss strategies:** Early training may benefit from combining GDL with cross-entropy ($\mathrm{Loss} = \alpha \cdot \mathrm{CE} + (1-\alpha)\cdot \mathrm{GDL}$ with $\alpha$ in $[0.2, 0.5]$) to stabilize optimization.
- **Monitoring:** Per-class Dice scores should be tracked throughout training to avoid collapse on rare classes.
- **Data augmentation:** Recommended especially for high anatomical variability, to prevent overfitting the class-weighted overlap criterion.

## 7. Comparative Performance and Qualitative Findings

GDL outperformed standard Dice and sensitivity–specificity losses in recovering small, “punctate” structures and reducing false negatives, especially on challenging 3D cases. Networks trained with standard Dice or sensitivity–specificity loss tended to miss fine lesion details or yield overly smooth predictions. By contrast, GDL-optimized models produced more accurate and detailed segmentation masks, exhibiting higher fidelity in recovering rare pathological structures even in the presence of extreme class imbalance [1707.03237].

Source: https://www.emergentmind.com/topics/generalised-dice-loss-gdl