---
title: 'EGAD: Entropy-Guided Sampling for Medical Segmentation'
url: https://www.emergentmind.com/topics/entropy-guided-agreement-diversity-egad
type: topic
---

# EGAD: Entropy-Guided Sampling for Medical Segmentation

Entropy-Guided Agreement-Diversity (EGAD) is a two-stage active learning framework that integrates predictive-entropy-based uncertainty sampling with sample diversity and agreement metrics for the targeted selection of data in semi-supervised learning (SSL) regimes. Primarily developed for label-efficient segmentation tasks in medical imaging, such as fetal head localization in ultrasound, EGAD aims to maximize model generalization with minimal annotation effort by strategically augmenting labeled datasets through an entropy-guided, agreement-diversity-driven query process [2601.17460].

## 1. Formal Problem Setting and Motivation

EGAD operates in the context of pool-based active learning for pixel-wise segmentation. Given an image pool $X = \{x_1,\ldots,x_N\}$ with a small initial labeled subset $L \subset X$ and a large unlabeled remainder $U = X \setminus L$, the objective is to iteratively improve a segmentation model $f(\cdot;\theta)$ by selectively annotating batches from $U$, subject to a strict annotation budget $B$. The segmentation problem is cast as binary pixel classification (foreground fetal head vs. background). Standard random or entropy-only sampling in SSL often leads to redundancies in labeled data and overfitting on less informative samples, motivating a more principled, diversity-aware acquisition protocol [2601.17460].

## 2. Two-Stage Active Learning Sampler

### 2.1 Predictive-Entropy Based Uncertainty Filtering

In the first stage, EGAD quantifies uncertainty by computing the total predictive entropy for each candidate image under the current model:
\[
H_j = -\sum_{c=1}^{2}\sum_{h=1}^{H}\sum_{w=1}^{W} p_{j,c,h,w} \log p_{j,c,h,w}
\]
where $p_{j,c,h,w}$ is the softmax probability at pixel $(h,w)$ for class $c$. The images in $U$ with the highest third of $H_j$ values are retained as the candidate pool $C_i$ for further screening [2601.17460].

### 2.2 Agreement-Diversity Scoring

A refined selection uses agreement-diversity criteria to encourage label diversity and minimize redundancy:

- **Agreement (Cosine Similarity):** For each $x_u \in C_i$, compute the average cosine similarity between its deep feature representation $z(x_u)$ and all labeled samples,
  \[
  \mathrm{CosSim}(x_u) = \mathrm{mean}_{x_l \in L_i}\left( \frac{z(x_l)^T z(x_u)}{\|z(x_l)\| \|z(x_u)\|} \right)
  \]
- **Diversity (Empirical Mutual Information):** For raw pixel intensities,
  \[
  \mathrm{MI}(x_u) = \mathrm{mean}_{x_l \in L_i} \sum_{i,j} p_{X_uX_l}(i,j) \log \left[\frac{p_{X_uX_l}(i,j)}{p_{X_u}(i)\, p_{X_l}(j)}\right]
  \]
  where $p_{X_uX_l}(i,j)$ is the joint histogram.

Both $\mathrm{CosSim}$ and $\mathrm{MI}$ are normalized to $[0,1]$ across $C_i$. The final composite score is:
\[
\mathrm{Score}(x_u) = \mathcal{N}(\mathrm{CosSim}(x_u)) - \mathcal{N}(\mathrm{MI}(x_u))
\]
Images with high score (low agreement and high diversity) are selected for annotation, ensuring the labeled set captures underrepresented regions of input space [2601.17460].

## 3. Consistency-Based Semi-Supervised Learning with Feature Downsampling

EGAD incorporates a co-training paradigm with dual models—one CNN (UNeXt) and one transformer (Swin-Unet)—each leveraging both labeled and unlabeled samples. The SSL loss is:
\[
L_{\text{total}} = L_{\mathrm{sup}}^1 + L_{\mathrm{sup}}^2 + \lambda(t)(L_{\mathrm{semi}}^1 + L_{\mathrm{semi}}^2) + L_{\mathrm{con}}
\]
Pseudo-labels from one network supervise the other on unlabeled data (cross pseudo-supervision). The feature consistency term $L_{\mathrm{con}}$ applies $L_2$-normalized, downsampled deep features to emphasize foreground structure while suppressing background noise. Gaussian noise perturbation with $\sigma=0.2$ is used for data augmentation in the consistency terms [2601.17460].

## 4. EGAD Training Pipeline

The end-to-end pipeline operates as follows:

1. Initialize $L_0$ (5−10% randomly labeled), $U_0 = X \setminus L_0$.
2. For each round $i$:
    - Train both networks jointly using $L_{\mathrm{total}}$ on $(L_i, U_i)$.
    - Infer entropy and feature statistics over $U_i$.
    - Construct $C_i$ (top third entropy).
    - Compute agreement-diversity scores in $C_i$, select batch $S_i$.
    - Label $S_i$ via oracle and update $L_{i+1} = L_i \cup S_i$, $U_{i+1} = U_i \setminus S_i$.
3. Terminate at budget exhaustion, deploy final CNN (UNeXt) [2601.17460].

## 5. Empirical Evaluation and Results

EGAD was rigorously validated on HC18 and ES-TCB fetal head ultrasound datasets. For training label fractions as low as 5–10%, SSL-EGAD consistently outperformed random/entropy-only and previous semi-supervised methods:

| Method                  | Labeled (%) | HC18 Dice (%) | ES-TCB Dice (%) | HC18 HD | ES-TCB HD |
|-------------------------|-------------|:-------------:|:---------------:|:-------:|:---------:|
| Best prior SSL (LCMT)   | 5           | 93.24         | 92.70           | 46.90   | 47.38     |
| Entropy (CEAL)          | 5           | 94.07         | 94.50           | 41.13   | 48.82     |
| Random                  | 5           | 94.06         | 91.84           | 35.87   | 57.59     |
| **SSL-EGAD (Ours)**     | 5           | **95.09**     | **94.04**       | 28.77   | 41.43     |
| Best prior SSL (LCMT)   | 10          | 95.31         | 95.15           | 29.50   | 39.32     |
| **SSL-EGAD (Ours)**     | 10          | **96.44**     | **96.21**       | 19.15   | 25.74     |

The method is robust across gestational stages, with performance gains persisting in second and third trimester test splits [2601.17460].

## 6. Entropy and Diversity Indices: Broader Context

Broader research establishes the link between ensemble diversity (including structural or predictive diversity) and generalization. Shannon entropy, Simpson’s index, and the Berger–Parker index provide orthogonal mechanisms to quantify diversity:
- Shannon entropy measures uncertainty as $H = -\sum_{i=1}^M p_i \log p_i$, normalized by $\log M$.
- Simpson’s index defines diversity as $D_{\mathrm{Simpson}} = 1 - \sum_{i=1}^M p_i^2$.
- Berger–Parker focuses on class dominance: $\mathrm{BP} = \max_{i} p_i$.

Empirical studies indicate that ensembles or labeled sets with higher normalized Shannon or Simpson diversity yield superior accuracy, whereas extreme evenness (maximum diversity) may induce slight declines, and dominance-based (Berger–Parker) indices are weak predictors. For EGAD frameworks, using normalized entropy or Simpson indices with agreement terms (e.g., cosine similarity) facilitates a principled tradeoff between uncertainty-driven informativeness and diversity-driven generalizability [0810.3525].

## 7. Limitations and Prospective Extensions

EGAD is validated exclusively for 2D fetal head segmentation, and the diversity score's mutual-information term leverages a fixed downsampling operator. Statistical significance of performance gains is not formally reported. Prospective improvements include extending to more anatomical structures (e.g., cerebellum, femur), 3D ultrasound, adopting advanced diversity selectors (e.g., sub-modular optimization or Bayesian entropy), and exploring adaptive budget allocation throughout active learning cycles [2601.17460].

Source: https://www.emergentmind.com/topics/entropy-guided-agreement-diversity-egad