A survey of loss functions for semantic segmentation (2006.14822v4)

Abstract: Image Segmentation has been an active field of research as it has a wide range of applications, ranging from automated disease detection to self-driving cars. In the past five years, various papers came up with different objective loss functions used in different cases such as biased data, sparse segmentation, etc. In this paper, we have summarized some of the well-known loss functions widely used for Image Segmentation and listed out the cases where their usage can help in fast and better convergence of a model. Furthermore, we have also introduced a new log-cosh dice loss function and compared its performance on the NBFS skull-segmentation open-source data-set with widely used loss functions. We also showcased that certain loss functions perform well across all data-sets and can be taken as a good baseline choice in unknown data distribution scenarios. Our code is available at Github: https://github.com/shruti-jadon/Semantic-Segmentation-Loss-Functions.

• The paper introduces a novel log-cosh dice loss function to smooth optimization and enhance segmentation training.
• It systematically categorizes 15 loss functions into four groups, improving clarity in function performance evaluation.
• Experimental results on a skull segmentation dataset demonstrate the efficacy of approaches like Focal Tversky Loss with high Dice coefficients.

A Survey of Loss Functions for Semantic Segmentation: An Expert Overview

The paper "A survey of loss functions for semantic segmentation" by Shruti Jadon provides an in-depth examination of various loss functions for semantic segmentation, a crucial task in image analysis with significant applications in fields such as medical imaging and autonomous vehicles. The work not only reviews the state-of-the-art loss functions but also introduces a novel log-cosh dice loss function, demonstrating its efficacy on a skull segmentation dataset. This essay aims to distill the essential findings and contributions of the paper for an audience of experienced researchers.

Overview and Classification of Loss Functions

At the core, the paper organizes fifteen widely used loss functions into four categories: Distribution-based, Region-based, Boundary-based, and Compounded. This categorization aids in systematically evaluating the effectiveness of each function depending on the segmentation task at hand.

1. Distribution-based Loss Functions:
   • Binary Cross-Entropy (BCE) and its variants like Weighted BCE and Balanced BCE are derived from probabilistic distributions. These are effective in handling data with class imbalances by introducing weighting mechanisms.
   • Focal Loss, a variant of BCE, focuses on down-weighting well-classified examples and puts more emphasis on hard, often misclassified examples.
2. Region-based Loss Functions:
   • Dice Loss and its variants, such as Tversky Loss and Focal Tversky Loss, are based on measures like the Dice Coefficient. These are particularly useful for tasks where small regions are of significant interest, providing better performance in highly imbalanced datasets.
3. Boundary-based Loss Functions:
   • Hausdorff Distance Loss and Shape-aware Loss target the boundaries of segmentation, taking into account the spatial arrangement of the segmented regions.
4. Compounded Loss Functions:
   • Combo Loss and Exponential Logarithmic Loss combine different loss components to leverage the strengths of each, thereby achieving superior performance in various scenarios.

Introduction of Log-Cosh Dice Loss

The paper introduces the log-cosh dice loss function, aiming to mitigate the non-convexity issues of the Dice loss by integrating the log-cosh approach known from regression problems. The log-cosh function is continuous and finite, making it computationally more stable and smoothing the optimization landscape.

Experimental Comparisons

An extensive experimental evaluation was conducted on the NBFS skull-stripping dataset using a 2D U-Net architecture. The research compared the performance of several loss functions using well-established evaluation metrics: Dice Coefficient, Sensitivity, and Specificity. Key findings include:

• Focal Tversky Loss achieved the highest Dice Coefficient (~0.98) and sensitivity, making it particularly effective for imbalanced datasets.
• The newly proposed log-cosh dice loss also exhibited a high Dice Coefficient (~0.975), demonstrating its potential as a reliable alternative.

Practical and Theoretical Implications

Practically, the paper provides a comprehensive guide for selecting appropriate loss functions based on dataset characteristics and segmentation requirements. Theoretically, it contributes a robust alternative in the form of the log-cosh dice loss, promising enhanced optimization performance.

Future Directions

Future developments could include the application of these findings in few-shot segmentation settings where labeled data is scarce. The ability to generalize over different datasets and maintain performance with minimal training data would further extend the utility of these loss functions.

In conclusion, Jadon's paper offers a thorough review and novel contributions to the field of semantic segmentation loss functions, providing valuable insights and tools for the development and optimization of deep learning models in image analysis.