Generalized Dice Focal Loss for Medical Segmentation

• GDFL is a composite loss function that integrates Generalized Dice Loss for region overlap with Focal Loss for emphasizing hard-to-classify voxels.
• It effectively mitigates extreme class imbalance in PET/CT imaging, improving segmentation accuracy for small and low-contrast lesions.
• Empirical performance on 3D Residual UNet models shows enhanced recall and stability, providing actionable insights for medical imaging research.

Generalized Dice Focal Loss (GDFL) is a composite objective function designed to address the challenges of extreme class imbalance and hard-to-classify structures in medical image segmentation, particularly for volumetric tasks such as lesion segmentation in PET/CT imaging. It is constructed as a straightforward sum of the Generalized Dice Loss (GDL)—which provides region-based overlap optimization robust to class imbalance—and the Focal Loss (FL), which concentrates learning on hard-to-classify (often rare, foreground) voxels. This combination targets the training instability and low recall typically observed for small-object segmentation in highly imbalanced data, such as volumetric cancer lesion detection in whole-body PET/CT images (Ahamed, 16 Sep 2024, Ahamed et al., 2023).

1. Mathematical Formulation

The binary GDFL is formally expressed as: $$\mathcal{L}_{\mathrm{GDFL}} = \mathcal{L}_{\mathrm{GDL}} + \mathcal{L}_{\mathrm{FL}}$$ where, within a minibatch of $n_b$ cubic patches (size $N^3$ each), with classes $l \in \{0,1\}$ (background, foreground):

• $p_{ilj}$: raw network logit for voxel $j$ in patch $i$, class $l$.
• $g_{ilj} \in \{0,1\}$: corresponding ground-truth.

Generalized Dice Loss: $$\mathcal{L}_{\mathrm{GDL}} = 1 - \frac{1}{n_b} \sum_{i=1}^{n_b} \frac{ \displaystyle \sum_{l=0}^1 w_{il} \sum_{j=1}^{N^3} p_{ilj} g_{ilj} + \epsilon }{\displaystyle \sum_{l=0}^1 w_{il} \sum_{j=1}^{N^3} (p_{ilj} + g_{ilj}) + \eta }$$ with class weights

$w_{il} = \frac{1}{\left(\sum_j g_{ilj}\right)^2}$

and smoothing constants $\epsilon = \eta = 10^{-5}$.

Focal Loss: $$\mathcal{L}_{\mathrm{FL}} = - \frac{1}{n_b} \sum_{i=1}^{n_b} \sum_{l=0}^1 \sum_{j=1}^{N^3} v_l [1 - \sigma(p_{ilj})]^\gamma\, g_{ilj}\, \log(\sigma(p_{ilj}))$$ where $\sigma(x) = 1/(1+e^{-x})$ is the sigmoid activation, $v_0 = 1$ (background), $v_1 = 100$ (foreground), and $\gamma = 2$ (Ahamed, 16 Sep 2024, Ahamed et al., 2023).

The GDFL, therefore, integrates class-level volume normalization, smoothing for numerical stability, and severe up-weighting of foreground voxel errors to directly address the core obstacles in medical volumetric segmentation.

2. Relationship to Standard Dice, Focal, and Hybrid Losses

Standard Dice Loss, while mitigating imbalance compared to cross-entropy, treats all voxels and classes equally, leading to domination by background when lesions occupy small volumes. Generalized Dice Loss addresses this by incorporating per-class inverse-squared volume weighting, strongly up-weighting underrepresented lesion voxels.

Focal Loss, originally introduced for object detection, modulates the loss contribution by a focusing factor—down-weighting the loss from easy, well-classified voxels and thus emphasizing hard, frequently misclassified voxels. In GDFL, the Focal Loss term is augmented with aggressive foreground up-weighting ($v_1 = 100$).

The summation of these components in GDFL yields complementary regularization: the Dice component enforces region-wise overlap for robust global volume prediction, while the Focal component sharpens boundary detection and boosts learning signals from sparse, ambiguous, or low-contrast lesion voxels (Ahamed, 16 Sep 2024).

This approach is distinct from parametric or interpolated Dice-CE mixtures (as in Unified Focal Loss (Yeung et al., 2021)) by employing a fixed, additive combination and empirically selected, static weights, with aggressive focal scaling and no explicit interpolation factor ($\alpha$ or $\lambda$).

3. Hyperparameter Choices and Design Rationale

The key hyperparameters of the GDFL, as employed in recent studies, are:

Component	Value	Rationale
Patch size	$N = 128$ or $192$	Trade-off between context and GPU memory
Minibatch size	$n_b = 4$	Implicit gradient stabilization; batch-norm effectiveness
GDL class weights	$w_{il} = 1/\mathrm{Vol}_{l, i}^2$	Strongly up-weight rare (foreground) voxels
Focal Loss weights	$v_0 = 1, v_1 = 100$	Empirically offsets background/foreground imbalance
Focusing parameter	$\gamma = 2$	Standard in Focal Loss literature; empirically stable
Smoothing	$\epsilon = \eta = 10^{-5}$	Prevents division-by-zero for absent classes in patch

These selections are motivated by the high class imbalance in PET/CT lesions. The foreground weight of 100 for the Focal Loss term, derived empirically, ensures sufficient gradient on scarce lesion voxels. The absence of an explicit combination weight ($\alpha$) between the loss terms is justified by the stabilization observed in experiments (Ahamed, 16 Sep 2024, Ahamed et al., 2023).

4. Implementation Practices and Numerical Stability

The reference implementation is based on 3D Residual UNet backbones. Key steps include:

• Patch extraction: Random cubic patches of $128^3$ (or $192^3$) voxels per batch; each batch consists of four patches.
• On-the-fly weight computation: Class weights $w_{il}$ calculated per patch, directly from the ground-truth mask, to handle intra-batch variability in foreground prevalence.
• Inference protocol: Sliding-window ($192^3$, 50% overlap), with voxelwise logits averaged across overlaps.
• Sigmoid mapping: For the Dice term, logits are processed through a sigmoid before probability overlap calculation. For the Focal term, the sigmoid output is directly used.
• Optimization: Adam optimizer ($10^{-3}$ initial learning rate), reduced to zero using cosine annealing over 400 epochs. The small smoothing constants are critical to prevent undefined values, especially in rare-foreground patches.
• No explicit loss scaling: The Dice and Focal terms are summed directly without rebalancing, as foreground scaling in the Focal term suffices (Ahamed, 16 Sep 2024).

5. Empirical Performance and Comparative Impact

Cross-validation of GDFL-trained 3D Residual UNet models on the AutoPET Challenge 2024 data yielded per-fold Dice Similarity Coefficients (DSC) in the range 0.55–0.63 and a mean DSC of 0.6687 on preliminary held-out test data, with a mean false negative volume (FNV) of 10.9522 ml and mean false positive volume (FPV) of 2.9684 ml (Ahamed, 16 Sep 2024). Earlier work demonstrated a 3–5% absolute DSC improvement when adding the focal term to a Generalized Dice baseline on similar PET/CT data (Ahamed et al., 2023).

GDFL exhibits particular benefit for small or low-contrast lesions, maintaining higher recall where standard Dice-based approaches tend to ignore tiny connected components and where baseline methods (e.g., nnU-Net) underperform on sparse foreground queries.

While explicit ablation (Dice-only, Focal-only, Dice+CE) is not reported in (Ahamed, 16 Sep 2024), prior studies substantiate that the joint loss offers stability advantages and improved quantitative metrics for highly imbalanced medical segmentation.

6. Practical Insights, Limitations, and Recommendations

The principal strengths of GDFL are robust suppression of easy negatives (via highly weighted Focal Loss) and effective handling of foreground sparsity using per-patch inverse-volume weighting in the Dice term. The implementation is compatible with most segmentation architectures using per-voxel loss accumulation.

Several caveats attend deployment:

• The foreground weight $v_1 = 100$ is empirically tuned for PET/CT data and may require adaptation for other datasets or less extreme class ratios.
• The effectiveness of the smoothing factors ($10^{-5}$) depends on floating-point precision and patch size; inappropriate values may induce numerical instability.
• GDFL does not explicitly penalize object-level statistics (e.g., instance false positives/negatives), and does not integrate direct control over FPV or FNV; modifications would be required for explicit volumetric penalties.
• Convergence is slower compared to standard cross-entropy; extended training (up to 400 epochs) and learning rate decay are necessary.

The GDFL hyperparameters ($\gamma = 2, v_1 = 100, \epsilon = \eta = 10^{-5}$) are established as effective for whole-body PET/CT segmentation, but should be re-validated for other modalities, anatomical regions, or task paradigms (Ahamed, 16 Sep 2024, Ahamed et al., 2023).

7. Relation to Broader Loss Frameworks

GDFL resides in a broader family of hybrid objective functions targeting medical segmentation class imbalance, including the Unified Focal Loss (Yeung et al., 2021), which provides a hierarchical parametrization encompassing CE, Dice, Focal, and Tversky as special cases. The distinguishing aspect of GDFL is its fixed, additive construction and direct, empirically motivated weighting regime, which contrasts with the interpolation and class-asymmetry scheme in Unified Focal Loss. Both approaches affirm the necessity of combining region-level overlap and voxel-level hard-mining penalties in extreme imbalance regimes, with specific parameterizations providing task-dependent benefit (Ahamed, 16 Sep 2024, Yeung et al., 2021).

---

GDFL thus represents a concise, empirically validated strategy for segmentation in highly imbalanced volumetric regimes, with demonstrated efficacy for PET/CT lesion analysis and a pathway for adaptation to related class-rare medical imaging contexts.