Consistency of Soft-IoU/Soft-Dice Loss for Semantic Segmentation

Determine whether the soft-IoU and soft-Dice surrogate loss functions used for training semantic segmentation models are calibrated (consistent) with respect to the Intersection over Union (IoU) and Dice metrics, i.e., whether minimizing these losses yields predictions that are Bayes-optimal for IoU and Dice evaluation.

Background

The paper reviews surrogate losses that aim to directly optimize IoU and Dice, highlighting that Lovász hinge losses have been shown to be inconsistent, thereby casting doubt on surrogate-based approaches for segmentation metric optimization. In contrast, soft-IoU and soft-Dice losses are widely used but non-convex, raising both optimization and theoretical concerns.

Within this context, the authors explicitly state that the consistency of soft-IoU/Dice losses remains unclear. Establishing their consistency (calibration) would clarify whether these surrogate objectives lead to Bayes-optimal predictions for IoU and Dice, aligning training objectives with evaluation metrics and informing the theoretical foundations of segmentation methods.