---
title: 'DSM Dice: Dual-Sampling Modulated Dice Loss'
url: https://www.emergentmind.com/topics/dual-sampling-modulated-dice-loss-dsm-dice
type: topic
---

# DSM Dice: Dual-Sampling Modulated Dice Loss

Dual-sampling Modulated Dice Loss (DSM Dice) is a segmentation loss formulation designed to address the challenges of extremely unbalanced and size-variable object segmentation, specifically in the context of hard exudate segmentation from color fundus images. DSM Dice introduces a dual-branch learning framework and enforces adaptive, epoch-wise modulation between two branches biased toward large and small exudates, respectively. The approach combines uniform and re-balanced pixel sampling in the Dice loss, allowing for simultaneous optimization of both high sensitivity to small lesions and high precision on large lesions [2012.01665].

## 1. Mathematical Formulation

DSM Dice defines three central loss terms involving soft segmentation predictions $\hat{Y} \in [0,1]^{H \times W}$ and binary masks $Y \in \{0,1\}^{H \times W}$, using pixel multisets $\mathcal{U}$ (uniform sampled) and $\mathcal{R}$ (re-balanced sampled), each of cardinality $N = H \cdot W$. For any multiset $\mathcal{S}$, the Dice score is defined as:
$$
\mathrm{DSC}(\hat Y[\mathcal{S}], Y[\mathcal{S}]) = \frac{2 \cdot \sum_{i \in \mathcal{S}} \hat y_i y_i}{\sum_{i \in \mathcal{S}} \hat y_i + \sum_{i \in \mathcal{S}} y_i}
$$
with loss $L_{\mathrm{dice}}(\mathcal{S}) = 1 - \mathrm{DSC}(\hat Y[\mathcal{S}], Y[\mathcal{S}])$.

**(1) Uniform-sampling Dice loss**:
$$
L_{\mathrm{dice}}^{(u)} = 1 - \frac{2 \sum_{i \in \mathcal{U}} \hat y_i y_i}{\sum_{i \in \mathcal{U}} \hat y_i + \sum_{i \in \mathcal{U}} y_i}
$$

**(2) Re-balanced-sampling Dice loss**:
Given $N_1$ exudate pixels and $N - N_1$ background pixels per batch, sampled with replacement,
$$
L_{\mathrm{dice}}^{(r)} = 1 - \frac{2 \sum_{i \in \mathcal{R}} \hat y_i y_i}{\sum_{i \in \mathcal{R}} \hat y_i + \sum_{i \in \mathcal{R}} y_i}
$$

**(3) Final modulated loss**:
With epoch-dependent mixing coefficient $\alpha(t) \in [0,1]$,
$$
L_{\mathrm{DSM}}(t) = (1 - \alpha(t)) \cdot L_{\mathrm{dice}}^{(u)} + \alpha(t) \cdot L_{\mathrm{dice}}^{(r)}
$$

$\varepsilon \approx 10^{-6}$ is introduced to denominators for numerical stability.

## 2. Pixel Sampling Schemes

DSM Dice loss employs two distinct multiset-based sampling distributions per image for Dice loss computation:

- **Uniform sampler**: Each of the $N$ pixel indices in $\mathcal{U}$ is drawn independently with probability $1/N$ from $\{1,\ldots,H \cdot W\}$, disregarding class balance.
- **Re-balanced sampler**: The sampling rate $r = N_1 / N$ governs the ratio of exudate to background pixels. For $P = \{i \mid Y_i=1\}$ and $B = \{i \mid Y_i=0\}$, for $n_p = |P|, n_b = |B|$,
    - Each exudate pixel $i \in P$ is sampled with $w_i = r / n_p$,
    - Each background pixel $i \in B$ is sampled with $w_i = (1 - r) / n_b$,
    - $\mathcal{R}$ is drawn i.i.d. according to these weights.

The uniform scheme biases optimization toward large exudates (due to prevalence in pixel count), while the re-balanced scheme compensates for the under-representation of small exudates.

## 3. Easy-to-Difficult Modulation Schedule

DSM Dice transitions the learning focus from predominantly large exudate segmentation to improved sensitivity for small exudates by applying a quadratic decay schedule to the mixing coefficient:
$$
\alpha(t) = 1 - (t/T)^2
$$
with $t$ the current epoch and $T$ the total training epochs. At epoch $t=0$, full emphasis is on the uniform loss, favoring large exudates. As training progresses, increasing weight is given to the re-balanced loss, prioritizing small-exudate segmentation.

## 4. Training Workflow and Pseudocode

Training is conducted with a dual-branch network sharing feature extraction, with each branch paired to a respective sampling and loss computation:

```python
# Inputs: batch {X}, {Y}; epoch t; total epochs T; sample rate r=N1/N

for X in batch:
    # (a) Uniform-biased branch
    Ŷ_u = Model_shared(X)
    # (b) Re-balanced-biased branch
    Ŷ_r = Model_shared(X)

L_u_total, L_r_total = 0, 0
for Ŷ_u, Y in zip(batch_preds, batch_gts):
    N = H * W
    U = sample_indices_uniformly(N, N)
    R = sample_indices_rebalanced(N, N, r, Y)
    L_u = 1 - 2*sum(Ŷ_u[U]*Y[U]) / (sum(Ŷ_u[U]) + sum(Y[U]) + ε)
    L_r = 1 - 2*sum(Ŷ_r[R]*Y[R]) / (sum(Ŷ_r[R]) + sum(Y[R]) + ε)
    L_u_total += L_u
    L_r_total += L_r

L_u_mean = L_u_total / batch_size
L_r_mean = L_r_total / batch_size
α = 1 - (t/T)**2
L_DSM = (1-α)*L_u_mean + α*L_r_mean
L_DSM.backward()
optimizer.step()
```
Here, the uniform branch specializes in detection of large exudates, while the re-balanced branch improves sensitivity to small exudates through targeted sampling.

## 5. Hyper-parameter Configuration and Implementation Guidelines

Key parameters and procedural aspects influencing DSM Dice efficacy include:

- **Sample-rate $r$**: The fraction of exudate pixels in the re-balanced sampler. Empirical ablation (Table 1) identifies $r=0.5$ as optimal, with $r \rightarrow 0$ or $r \rightarrow 1$ degenerating to single-branch training.
- **Epochs $T$**: Controls the speed of modulation from large- to small-exudate bias. Quadratic decay for $\alpha(t)$ is smoother than linear.
- **Numerical stabilization**: Inclusion of a small constant $\varepsilon$ in denominators.
- **Optimization**: Uses standard practices, such as polynomial-decay learning rates and SGD.

A summary of key settings is provided below:

| Parameter          | Recommended Value/Practice         | Effect                                     |
|--------------------|-----------------------------------|--------------------------------------------|
| Sample-rate $r$    | 0.5                               | Balances large- and small-lesion learning  |
| Decay schedule     | $\alpha(t) = 1-(t/T)^2$           | Smooth transition; favors late sensitivity |
| $\varepsilon$      | $\sim 10^{-6}$                    | Stabilizes denominator in Dice computation  |
| Optimizer config   | Poly-LR decay, SGD                | Standard segmentation settings             |

## 6. Ablation Studies and Comparative Impact

DSM Dice loss was benchmarked on the public DDR dataset using dual-PSPNet architectures. Core findings include:

- **Sample-rate effect**:
    - $r=0.25$: IoU=0.4069, AUPR=0.5468
    - $r=0.50$: IoU=0.4103, AUPR=0.5587 (optimal)
    - $r=0.75$: IoU=0.4068, AUPR=0.5491
    - $r$ as inverse frequency: IoU=0.3898, AUPR=0.5202
- **Branch comparisons**:
    - Single-branch Dice (uniform only) under-segments small exudates.
    - Single-branch CBCE loss over-segments background.
    - Dual-branch + DSM yields high sensitivity for small lesions, high precision for large lesions.
    - Dual-PSPNet with DSM: IoU increases from 0.3822 (single Dice) to 0.4103; AUPR from 0.4730 to 0.5587.
    - Region-level F-score (σ=0.5): single PSPNet+Dice 0.6954, dual-PSPNet+DSM 0.7543.

These results demonstrate that DSM Dice achieves state-of-the-art performance by leveraging branch-specialized optimization and adaptive loss modulation, benefiting both pixel- and region-level segmentation [2012.01665].

## 7. Context and Significance in Medical Image Segmentation

DSM Dice was introduced to address the fundamental limitation of uniform pixel-wise Dice loss, which under-weights small but clinically important patterns in heavily imbalanced segmentation tasks. The dual-branch, dual-sampling structure, combined with an easy-to-difficult modulation schedule, enables more robust recognition across object scales, particularly improving detection of small pathologies such as micro-lesions. This approach can be generalized to other segmentation contexts characterized by similar class imbalance and object size variance, where standard losses may fail to capture rare or subtle structures effectively.

Source: https://www.emergentmind.com/topics/dual-sampling-modulated-dice-loss-dsm-dice