Unified Focal Loss (UFL) Overview

• Unified Focal Loss is a unified loss framework that combines margin regularization and focal modulation to effectively weight hard and easy samples.
• It offers multiple instantiations, including margin-based, hierarchical region-based, and relative-error scaling strategies for tasks like segmentation and depth regression.
• UFL recovers popular loss functions as special cases, improving performance on imbalanced datasets while reducing hyperparameter complexity.

Unified Focal Loss (UFL) is a family of supervised loss functions extending the classical focal loss paradigm to a broader class of prediction problems and architectures by fusing distinct mechanisms for handling data imbalance and optimization difficulty. UFL has independently emerged in multiple forms in recent literature, with three notable instantiations: (1) a margin-regularized focal-weighted softmax loss for segmentation (Chen, 2023), (2) a hierarchical combination of distributional (focal) and region-based (Focal Tversky) losses for medical image analysis (Yeung et al., 2021), and (3) a relative-error-scaled focal loss for multi-view stereo depth regression with continuous targets (Peng et al., 2022). All variants share a unified goal: to jointly address hard/easy sample weighting and class/sample imbalance, and to serve as a generalization framework that recovers popular loss functions as special cases.

1. Mathematical Formulations and Core Principles

1.1 Margin-based Focal Loss (for Segmentation, Crack Detection)

Let $z_j$ denote the logit for class $j \in \{1, \dots, C\}$, $y$ the ground-truth class ($y \in \{1, \dots, C\}$), and $p_t = \frac{e^{z_y}}{\sum_{j=1}^C e^{z_j}}$ the standard softmax probability for the true class. The UFL loss in (Chen, 2023) introduces:

• Margin-Softmax Regularizer: For a fixed margin $m \geq 0$ and scaling factor $s>0$,

$$\hat z_j = \begin{cases} s\,(z_y + m), & j = y \ s\,z_j, & j \ne y \end{cases}$$

$$\hat p_t = \frac{\exp(s(z_y + m))}{\exp(s(z_y+m)) + \sum_{j\neq y} \exp(s z_j)} ,$$

$L_{\text{reg}} = -\log \hat p_t ;$

• Focal Modulation: For parameter $j \in \{1, \dots, C\}$0,

$j \in \{1, \dots, C\}$1

• Unified Focal Loss (convex combination):

$j \in \{1, \dots, C\}$2

1.2 Hierarchical Region and Distribution-Based UFL (Medical Segmentation)

Yeung et al. (Yeung et al., 2021) generalize focal and Dice-type losses under a single formulation:

• Modified Focal Loss:

$j \in \{1, \dots, C\}$3

• Modified Focal Tversky Loss:

$j \in \{1, \dots, C\}$4

$j \in \{1, \dots, C\}$5

• Unified Focal Loss (hierarchical):

$j \in \{1, \dots, C\}$6

1.3 Relative-Error Focal Loss for Continuous Targets (Multi-View Stereo)

In the context of continuous, sparse soft labels $j \in \{1, \dots, C\}$7, (Peng et al., 2022) proposes:

• Unified Focal Loss:

$j \in \{1, \dots, C\}$8

where $j \in \{1, \dots, C\}$9, $y$0, and $y$1 are bounded, sigmoid-like scaling functions.

2. Hyperparameters, Special Cases, and Tuning

Across these frameworks, UFL introduces a compact hyperparameter set, each governing a specific operational facet:

Parameter	Role	Typical/Recommended Values
$y$2 (margin)	Enforce class separation, rare-class robustness	$y$3 ($y$4 for class imbalance)
$y$5 (scale)	Amplify logits for effective margin application	$y$6 ($y$7)
$y$8 (focus)	Down-weight easy examples, focus on hard samples	$y$9 or $y \in \{1, \dots, C\}$0 (task dependent)
$y \in \{1, \dots, C\}$1 or $y \in \{1, \dots, C\}$2	Weight region/distribution balance	$y \in \{1, \dots, C\}$3 (pure margin: $y \in \{1, \dots, C\}$4; pure focal: $y \in \{1, \dots, C\}$5)
$y \in \{1, \dots, C\}$6	FP/FN trade-off in region-based losses	$y \in \{1, \dots, C\}$7
$y \in \{1, \dots, C\}$8 (scaling base)	Bounds error scaling in regressional UFL	$y \in \{1, \dots, C\}$9
$p_t = \frac{e^{z_y}}{\sum_{j=1}^C e^{z_j}}$0	Positive/negative sample weighting (regression)	$p_t = \frac{e^{z_y}}{\sum_{j=1}^C e^{z_j}}$1, $p_t = \frac{e^{z_y}}{\sum_{j=1}^C e^{z_j}}$2 stagewise (0.75→0.25)

Special cases include:

• Focal Loss: $p_t = \frac{e^{z_y}}{\sum_{j=1}^C e^{z_j}}$3, $p_t = \frac{e^{z_y}}{\sum_{j=1}^C e^{z_j}}$4 (Chen, 2023); $p_t = \frac{e^{z_y}}{\sum_{j=1}^C e^{z_j}}$5 with $p_t = \frac{e^{z_y}}{\sum_{j=1}^C e^{z_j}}$6, $p_t = \frac{e^{z_y}}{\sum_{j=1}^C e^{z_j}}$7 (Yeung et al., 2021)
• Margin Softmax: $p_t = \frac{e^{z_y}}{\sum_{j=1}^C e^{z_j}}$8, $p_t = \frac{e^{z_y}}{\sum_{j=1}^C e^{z_j}}$9 (Chen, 2023)
• Dice / Tversky: $m \geq 0$0, $m \geq 0$1, $m \geq 0$2 (Yeung et al., 2021)
• Binary FL: $m \geq 0$3 or $m \geq 0$4, $m \geq 0$5 in (Peng et al., 2022)

3. Implementation and Pseudocode

The UFL losses are directly compatible with common deep learning frameworks (TensorFlow/Keras, PyTorch). Below is a high-level pseudocode for the margin-focal variant in segmentation (Chen, 2023):

$$\hat p_t = \frac{\exp(s(z_y + m))}{\exp(s(z_y+m)) + \sum_{j\neq y} \exp(s z_j)} ,$$8

Similar pseudocode applies for hierarchical and continuous label forms (Yeung et al., 2021, Peng et al., 2022); computation remains plug-and-play with automatic differentiation.

4. Empirical Evaluation and Task-Specific Results

Image Segmentation and Crack Detection

In crack segmentation tasks (DeepCrack-DB, PanelCrack), the UFL with $m \geq 0$6 led to IoU gains of $m \geq 0$7 on DeepCrack-DB (from $m \geq 0$8 to $m \geq 0$9) and $s>0$0 on PanelCrack (Chen, 2023), illustrating the dual benefits on class imbalance (focal term) and rare-class overfitting (margin term).

Medical Image Analysis

In five medical segmentation datasets:

• CVC-ClinicDB (polyp), DSC improved to $s>0$1 (UFL, asymmetric), outperforming Dice, Focal, Tversky, and Combo losses (Yeung et al., 2021).
• Robust gains were observed in DRIVE, BUS2017, BraTS20, and KiTS19 across DSC and IoU, with all aUF improvements significant at $s>0$2.
• UFL maintained stable performance across $s>0$3 and reduced the hyperparameter burden compared to earlier hybrid losses.

Multi-View Stereo

For DTU depth estimation, the relative-error UFL (Peng et al., 2022) reduced overall error to $s>0$4 mm, outperforming both GFL and standard BCE. Ablation confirmed its robustness to scaling, hyperparameters, and stagewise weighting.

5. Mechanisms for Addressing Imbalance and Overfitting

UFL's empirical effectiveness stems from two core mechanisms, consistently present in all variants:

• Focal reweighting $s>0$5 (or its continuous generalizations) down-weights loss contributions from easy (i.e., high-confidence or background-dominated) examples, thus directing gradients toward rare (minority) classes. This mitigates extreme foreground/background imbalances in pixel- or hypothesis-dense tasks.
• Margin regularization (when included) further distances rare-class logits from the decision boundary, reducing overfitting and increasing robustness to annotation noise or scarce class occurrences.

In region-based UFL, the interplay of distributional and spatial (Tversky) terms ensures both pixelwise and regional balancing. Relative-error modulation in continuous targets (depth, keypoints) further amplifies errors on “hard” outliers, preventing loss domination by vast numbers of background or trivial samples.

6. Unification of Loss Function Families

UFL provides a principled framework that subsumes traditional loss functions:

Setting	Recovers
$s>0$6, $s>0$7	Cross-entropy
$s>0$8, $s>0$9	Margin softmax/ArcFace
$$\hat z_j = \begin{cases} s\,(z_y + m), & j = y \ s\,z_j, & j \ne y \end{cases}$$0, $$\hat z_j = \begin{cases} s\,(z_y + m), & j = y \ s\,z_j, & j \ne y \end{cases}$$1	Focal loss
$$\hat z_j = \begin{cases} s\,(z_y + m), & j = y \ s\,z_j, & j \ne y \end{cases}$$2, $$\hat z_j = \begin{cases} s\,(z_y + m), & j = y \ s\,z_j, & j \ne y \end{cases}$$3, $$\hat z_j = \begin{cases} s\,(z_y + m), & j = y \ s\,z_j, & j \ne y \end{cases}$$4 (Yeung et al., 2021)	Dice loss
$$\hat z_j = \begin{cases} s\,(z_y + m), & j = y \ s\,z_j, & j \ne y \end{cases}$$5, $$\hat z_j = \begin{cases} s\,(z_y + m), & j = y \ s\,z_j, & j \ne y \end{cases}$$6, $$\hat z_j = \begin{cases} s\,(z_y + m), & j = y \ s\,z_j, & j \ne y \end{cases}$$7	CE in Dice/focal-Dice
$$\hat z_j = \begin{cases} s\,(z_y + m), & j = y \ s\,z_j, & j \ne y \end{cases}$$8, $$\hat z_j = \begin{cases} s\,(z_y + m), & j = y \ s\,z_j, & j \ne y \end{cases}$$9 (Peng et al., 2022)	Standard Focal loss
GFL (Generalized Focal Loss) (Peng et al., 2022)	UFL w/o relative scaling

This precise recoverability ensures that UFL can flexibly adapt to the demands of specific architectures, data regimes, or evaluation needs.

7. Practical Considerations and Recommendations

• For imbalanced foreground-background tasks, select moderate to high $$\hat p_t = \frac{\exp(s(z_y + m))}{\exp(s(z_y+m)) + \sum_{j\neq y} \exp(s z_j)} ,$$0 ($$\hat p_t = \frac{\exp(s(z_y + m))}{\exp(s(z_y+m)) + \sum_{j\neq y} \exp(s z_j)} ,$$1–$$\hat p_t = \frac{\exp(s(z_y + m))}{\exp(s(z_y+m)) + \sum_{j\neq y} \exp(s z_j)} ,$$2), enforce a nonzero margin $$\hat p_t = \frac{\exp(s(z_y + m))}{\exp(s(z_y+m)) + \sum_{j\neq y} \exp(s z_j)} ,$$3 when overfitting is observed, and tune $$\hat p_t = \frac{\exp(s(z_y + m))}{\exp(s(z_y+m)) + \sum_{j\neq y} \exp(s z_j)} ,$$4 to balance region and distribution effects (Chen, 2023, Yeung et al., 2021).
• In continuous label and regression settings, maintain $$\hat p_t = \frac{\exp(s(z_y + m))}{\exp(s(z_y+m)) + \sum_{j\neq y} \exp(s z_j)} ,$$5 in $$\hat p_t = \frac{\exp(s(z_y + m))}{\exp(s(z_y+m)) + \sum_{j\neq y} \exp(s z_j)} ,$$6 within empirical ranges where relative-error scaling is effective but not prone to outlier explosion; set positive class weights to $$\hat p_t = \frac{\exp(s(z_y + m))}{\exp(s(z_y+m)) + \sum_{j\neq y} \exp(s z_j)} ,$$7 and decay negative weights or focusing parameters stage-wise as resolution increases (Peng et al., 2022).
• UFL integrates seamlessly with categorical, regression, and hybrid region-based objectives; no architectural modification is needed, only the loss function is replaced.

UFL, in its multiple instantiations, consistently improves performance in dense prediction tasks—particularly under conditions of severe class or sample imbalance—while maintaining interpretability and compatibility with standard training pipelines (Chen, 2023, Yeung et al., 2021, Peng et al., 2022).