Prototype Confidence-Modulated Thresholding (PCMT)

• Prototype Confidence-modulated Thresholding (PCMT) is a data-driven adaptive method that adjusts segmentation thresholds based on empirical prototype separation.
• It employs Otsu's method and a modulation function to balance false positives and precise boundary preservation in ambiguous segmentation tasks.
• Empirical results show consistent mIoU improvements across various domains, demonstrating its effectiveness especially in medical imaging.

Prototype Confidence-modulated Thresholding (PCMT) is a data-driven thresholding mechanism designed to mitigate segmentation ambiguity in cross-domain few-shot segmentation, particularly when query foreground and background regions exhibit high similarity in feature space. By adaptively adjusting the threshold for foreground assignment based on empirical prototype separation, PCMT addresses under- and over-segmentation challenges common in domains such as medical imaging, where lesion boundaries are often subtle and conventional prototype-matching approaches may yield substantial false positives.

1. Problem Context and Motivation

Cross-domain few-shot segmentation (CD-FSS) involves segmenting previously unseen classes in target domains that differ substantially from the source domain, using a minimal number of annotated samples. A persistent issue arises when query foreground and background features are highly similar, leading conventional prototype-matching schemes—typically based on comparing the cosine similarity between pixel features and foreground/background prototypes—to yield ambiguous and unreliable segmentation. Specifically, standard assignment rules of the form “pixel is foreground if $\cos(p^{fg}, F) > \cos(p^{bg}, F)$” can result in spurious foreground predictions or excessive erosion of object boundaries when prototype separation is poor.

PCMT addresses this by leveraging an adaptive threshold computed from the distribution of the confidence map, and further modulates this threshold using a prototype confidence score reflecting the degree of foreground-background prototype separation. This mechanism permits dynamic balancing between strict and lenient segmentation, contingent upon the confidence in prototype distinctiveness (Sun et al., 15 Nov 2025).

2. Mathematical Foundations

Let $\hat p_s^{fg}$ and $\hat p_s^{bg}$ denote support-set foreground and background prototypes, while $\hat p_q^{fg}$ and $\hat p_q^{bg}$ are query-set prototypes, obtained via self-support. The fused prototypes $\hat p^{fg}$ and $\hat p^{bg}$ incorporate information from both support and query sets. Given an upsampled query feature map $\hat F_q \in \mathbb{R}^{C \times H \times W}$, the method computes:

• Foreground similarity map: $M_q^{fg}(x,y) = \cos(\hat p^{fg}, \hat F_q(x,y))$
• Background similarity map: $M_q^{bg}(x,y) = \cos(\hat p^{bg}, \hat F_q(x,y))$
• Foreground confidence map: $M_q^{conf}(x,y) = M_q^{fg}(x,y) - M_q^{bg}(x,y)$

An adaptive threshold $t$ is calculated using Otsu’s method on the histogram of $M_q^{conf}$. The prototype confidence $C$ quantifies the effective separation, defined as:

$$C = \cos(\hat p_q^{fg}, \hat p_s^{fg}) - \frac{1}{2}\left[ \cos(\hat p_q^{fg}, \hat p_s^{bg}) + \cos(\hat p_s^{fg}, \hat p_q^{bg}) \right]$$

The threshold is then modulated by $C$ using:

$\tau(C) = \frac{1}{1+\exp(\beta(C+\gamma))} t$

where $\beta$ and $\gamma$ are hyperparameters. The final binary mask is obtained by thresholding:

$\widetilde M_q(x,y) = [M_q^{conf}(x,y) > \tau(C)]$

Notably, when $\tau(C) \rightarrow 0$ (high prototype confidence), the mechanism reduces to the original comparison-based rule, $M_q^{conf} > 0$.

3. Algorithmic Workflow

The following steps detail the PCMT procedure at inference time:

Step	Operation Description	Output/Usage
1	Compute query prototypes via SSP module	$p_q^{fg}, p_q^{bg}$
2	Fuse support/query prototypes	$p^{fg}, p^{bg}$
3	Upsample query feature map	$F_q$
4	Calculate similarity/confidence maps	$M_q^{fg}, M_q^{bg}, M_q^{conf}$
5	Otsu thresholding on $M_q^{conf}$	$t$
6	Compute prototype confidence $C$	$C$
7	Modulate threshold ($\tau$ computation)	$\tau(C)$
8	Generate mask via thresholding	$\widetilde M_q$

This procedure requires no extra learnable parameters and operates solely at inference, ensuring computational efficiency and transparency.

4. Implementation Characteristics

PCMT is implemented as a post-processing module following prototype matching and prior to any mask post-processing, such as conditional random fields (CRF) or morphological operations. The method utilizes $\ell_2$-normalized vectors for all cosine similarity calculations (consistent with the SSP module). Otsu's method leverages the per-image confidence map, operating without reliance on ground-truth data. Default hyperparameters are $\beta = 40.0$, $\gamma = 0.1$, and the Otsu bin count is set to 256. PCMT’s design ensures that no additional trainable components or model re-training are necessary—its effect is strictly due to the adaptive, data-driven modulation based on actual prototype separability at test time.

5. Empirical Performance and Observations

Ablation studies on four target-domain datasets (Deepglobe, ISIC, Chest X-ray, FSS-1000) reported consistent improvements in mean Intersection-over-Union (mIoU):

• ResNet-50 backbone: +1.26 mIoU (62.97 $\rightarrow$ 64.23)
• ViT-B/16 backbone: +1.19 mIoU (67.05 $\rightarrow$ 68.24)

Qualitatively, in highly ambiguous cases (e.g., ISIC skin lesion segmentation), adaptive thresholding moves away from zero, preventing inadvertent labeling of background pixels as foreground due to prototype similarity. In non-ambiguous cases (e.g., FSS-1000, which is near the source domain in feature space), prototype confidence $C$ is typically large, resulting in $\tau(C) \approx 0$ and recovery of the standard $M_q^{conf}>0$ assignment, thereby avoiding unnecessary erosion of object boundaries.

6. Significance and Implications

PCMT introduces a lightweight, fully inference-time solution to a persistent problem in cross-domain few-shot segmentation, balancing false-positive suppression and boundary preservation by modulating foreground assignment thresholds in relation to prototype distinctiveness. This suggests direct applicability to segmentation domains with naturally ambiguous foreground-background boundaries, such as medical imaging, remote sensing, and microscopy. A plausible implication is enhanced generalization to unseen classes in highly variable target domains, with minimal computational overhead or re-training requirements.

7. Integration with Hierarchical Semantic Learning Frameworks

In the context of the Hierarchical Semantic Learning (HSL) framework presented in "Bridging Granularity Gaps: Hierarchical Semantic Learning for Cross-domain Few-shot Segmentation" (Sun et al., 15 Nov 2025), PCMT complements modules such as Dual Style Randomization (DSR) and Hierarchical Semantic Mining (HSM) by specifically addressing the thresholding ambiguity endemic to prototype-based segmentation approaches. Its strategic placement ensures that improved semantic feature learning translates to superior pixel-level segmentation performance without incurring additional learnable complexity. PCMT exemplifies a trend toward simple, data-adaptive enhancements of classic prototype-matching strategies in the few-shot learning regime.