Semantic-Aware Threshold Learning (SATL)

• SATL is a threshold-learning method that estimates category-specific decision thresholds from the empirical score distributions of known positives and negatives in multi-label recognition.
• It dynamically updates thresholds using exponential smoothing, balancing pseudo-label precision and recall according to category-specific score behaviors.
• Empirical results on datasets like MS-COCO and VG-200 demonstrate that SATL significantly improves mAP and F1 metrics over global fixed-threshold baselines.

Searching arXiv for the cited SATL and related threshold-learning papers to ground the article in current papers. Searching arXiv for (Ruan et al., 31 Jul 2025). Semantic-Aware Threshold Learning (SATL) is a threshold-learning method for multi-label image recognition with partial labels (MLR-PL) that replaces heuristic pseudo-label cutoffs with category-specific thresholds estimated from score distributions and updated dynamically during training (Ruan et al., 31 Jul 2025). In this setting, each image may contain multiple categories, but only a subset of label entries is observed; SATL addresses the resulting pseudo-label selection problem by estimating, for each category, how model scores are distributed over known positive and known negative samples, then using those distributions to define a category-dependent decision threshold. The method is “semantic-aware” in a specific sense: it assumes that within a category, semantically similar samples exhibit similar prediction-score distributions, so known labels can inform thresholding for unknown labels (Ruan et al., 31 Jul 2025).

1. Problem formulation and conceptual basis

SATL is formulated for datasets

$D=\{(I^1, y^1), (I^2, y^2),...,(I^N, y^N)\},$

where $I^n$ is the $n$-th image and

$y^n=[ y^n_1, y^n_2, ..., y^n_C ] \in \{-1,0,1\}^C.$

For category $c$, $y_c^n = 1$ denotes presence, $y_c^n = -1$ denotes absence, and $y_c^n = 0$ denotes an unknown label. The central difficulty is that ignoring unknown labels wastes supervision, whereas treating them as negatives introduces false negatives. SATL therefore operates within a pseudo-labeling pipeline: a model first predicts category scores for unknown entries, and thresholding then determines which unknown labels are promoted to pseudo-positives (Ruan et al., 31 Jul 2025).

The method is motivated by the observation that categories exhibit different score distributions. Some categories are visually distinctive and yield well-separated positive and negative scores; others are ambiguous, long-tailed, or difficult, and their score distributions overlap. A single global threshold is therefore suboptimal. SATL addresses this by learning one threshold per category from the empirical score distributions of known positives and known negatives in that category, and by updating these thresholds as training progresses (Ruan et al., 31 Jul 2025).

In this formulation, the core pseudo-labeling rule is

$\widetilde{y}^n_c=1[\widetilde{p}^n_c > \tau_c],$

where $\widetilde{p}^n_c$ is the pseudo-label-generation score for category $I^n$0 in image $I^n$1, and $I^n$2 is the learned category-specific threshold. The method is therefore fundamentally a threshold-learning approach to positive-label mining under incomplete supervision.

2. Semantic-Aware Threshold Estimation

The threshold module in SATL is called Semantic-Aware Threshold Estimation (SATE). For category $I^n$3, SATL collects the output scores of known samples,

$I^n$4

and partitions them into known positives and known negatives according to the available labels. These two empirical score sets define the category’s positive and negative output distributions (Ruan et al., 31 Jul 2025).

The threshold rule is based on two uncertainty parameters, $I^n$5 and $I^n$6, through the quantile-style condition

$I^n$7

The paper then defines the ideal threshold for category $I^n$8 as

$I^n$9

If the positive and negative score distributions are well separated, this threshold lies near a low-density region and can admit more pseudo-labels; if the distributions overlap, taking the larger boundary yields a more conservative threshold, reducing false positives (Ruan et al., 31 Jul 2025).

Thresholds are not frozen after estimation. SATL updates them by exponential smoothing: $n$0 Here $n$1 is the current ideal threshold estimated from the latest score distributions, $n$2 is the running threshold, and $n$3 controls threshold momentum. The paper reports, for example, $n$4 at 20% known labels and $n$5 at 50% known labels. It also reports $n$6 and $n$7 in the main implementation, while noting that these parameters trade off pseudo-label precision and recall (Ruan et al., 31 Jul 2025).

A common misconception is that “semantic-aware” here implies an explicit external semantic graph, textual label embedding, or ontology inside the threshold estimator. The method does not do that. Rather, “semantic-aware” refers to the assumption that within a category, semantically similar samples generate similar prediction-score distributions, so known-labeled samples can approximate the score statistics of unknown-labeled ones. This is category semantics reflected in model outputs, not explicit graph-structured or language-conditioned semantic reasoning (Ruan et al., 31 Jul 2025).

3. Two-stage training and objective functions

SATL is a plug-in framework for pseudo-label-based MLR-PL methods such as SST and HST. Its pipeline has two stages. In stage 1, the model is trained only on known labels using a partial BCE objective. The paper states that the initial threshold is set to $n$8 during the first stage, effectively suppressing pseudo-label selection. In stage 2, the method repeatedly estimates category thresholds, generates pseudo-labels for unknown entries, fuses those pseudo-labels with known labels, and optimizes a combined objective with an auxiliary ranking loss (Ruan et al., 31 Jul 2025).

The base prediction module outputs

$n$9

while pseudo-label generation uses $y^n=[ y^n_1, y^n_2, ..., y^n_C ] \in \{-1,0,1\}^C.$0. The first-stage loss is

$y^n=[ y^n_1, y^n_2, ..., y^n_C ] \in \{-1,0,1\}^C.$1

with

$y^n=[ y^n_1, y^n_2, ..., y^n_C ] \in \{-1,0,1\}^C.$2

Unknown labels do not contribute to this objective (Ruan et al., 31 Jul 2025).

After thresholding, the fused target vector is

$y^n=[ y^n_1, y^n_2, ..., y^n_C ] \in \{-1,0,1\}^C.$3

and the modified classification loss becomes

$y^n=[ y^n_1, y^n_2, ..., y^n_C ] \in \{-1,0,1\}^C.$4

Pseudo-label generation is one-sided: $y^n=[ y^n_1, y^n_2, ..., y^n_C ] \in \{-1,0,1\}^C.$5 Unknown entries that do not cross threshold are not converted into pseudo-positives. The paper does not define an explicit negative pseudo-label assignment for unknown entries, so the mechanism is best understood as positive-label mining rather than explicit positive/negative/ignore triage (Ruan et al., 31 Jul 2025).

To increase separation between positive and negative score distributions, SATL adds a differential ranking loss (DRL). It first defines the threshold-relative exceedance

$y^n=[ y^n_1, y^n_2, ..., y^n_C ] \in \{-1,0,1\}^C.$6

The DRL is

$y^n=[ y^n_1, y^n_2, ..., y^n_C ] \in \{-1,0,1\}^C.$7

where

$y^n=[ y^n_1, y^n_2, ..., y^n_C ] \in \{-1,0,1\}^C.$8

The second-stage objective is then

$y^n=[ y^n_1, y^n_2, ..., y^n_C ] \in \{-1,0,1\}^C.$9

The intended effect is threshold-relative distribution separation: known positives are encouraged to lie above the threshold, while known negatives are penalized if they cross it (Ruan et al., 31 Jul 2025).

4. Empirical behavior, datasets, and ablations

SATL is evaluated on MS-COCO and VG-200, with partial labels simulated by retaining only 5%, 10%, 20%, 30%, 40%, or 50% of label entries per image. The reported metrics are mAP, OF1, and CF1. On MS-COCO, the average mAP values reported are 73.1 for SST, 74.5 for HST, 76.0 for SST+SATL, and 76.5 for HST+SATL. On VG-200, the corresponding average mAP values are 40.1 for SST, 42.6 for HST, 45.3 for SST+SATL, and 45.3 for HST+SATL. The paper states that paired t-tests on MS-COCO yield $c$0 for all comparisons between SATL variants and baselines (Ruan et al., 31 Jul 2025).

The reported F1 results follow the same pattern. On MS-COCO, SST gives OF1 72.9 and CF1 68.7, HST gives OF1 73.7 and CF1 69.7, SST+SATL gives OF1 75.4 and CF1 70.0, and HST+SATL gives OF1 76.0 and CF1 71.5. On VG-200, SST gives OF1 41.2 and CF1 32.0, HST gives OF1 40.6 and CF1 31.5, SST+SATL gives OF1 46.9 and CF1 40.4, and HST+SATL gives OF1 46.0 and CF1 39.5. The larger gain on VG-200 is presented as evidence that category-specific threshold learning becomes particularly useful when the task has more categories and is harder (Ruan et al., 31 Jul 2025).

Ablation studies separate the effects of SATE and DRL. On MS-COCO, average mAP is 73.1 for SST, 75.6 for “Ours SATE,” 75.3 for “Ours DRL,” and 76.0 for the full method. On VG-200, the averages are 40.1, 44.8, 44.2, and 45.3, respectively. This identifies SATE as the primary contributor, while also indicating that DRL improves distribution separation and complements threshold estimation (Ruan et al., 31 Jul 2025).

The paper also reports qualitative findings on threshold behavior. SATL increases pseudo-label recall while maintaining high pseudo-label precision, and its learned thresholds often lie near the low-density gap between positive and negative score distributions when these are separable, but become more conservative when the distributions overlap. For the 10 least-represented categories in MS-COCO, the paper reports average pseudo-label precision of 97.1% and recall of 33.2%, consistent with conservative thresholding for rare classes (Ruan et al., 31 Jul 2025).

5. Relation to adjacent threshold-learning methods

SATL belongs to a broader line of adaptive thresholding methods, but it occupies a distinct position. Fixed-threshold pseudo-labeling uses a single global cutoff. SATL instead learns one threshold per category from that category’s current score distributions. This makes it closer to distribution-driven category-specific thresholding than to globally scheduled thresholding (Ruan et al., 31 Jul 2025).

A useful contrast is SST, “Self-training with Self-adaptive Thresholding,” whose core mechanism is also abbreviated SAT but is not semantic-aware in the sense used by SATL. SST computes class-specific thresholds from class-wise confidence distributions on unlabeled data, using

$c$1

with thresholds recomputed only once per self-training cycle. Its adaptation signal is class-wise confidence statistics from a relatively mature model, not semantic relations, feature geometry, text descriptions, or a learned threshold network. As the paper’s own framing makes clear, SST is best understood as adaptive confidence thresholding with class-aware statistics rather than rich semantic-aware threshold learning (Zhao et al., 31 May 2025).

InstanT pushes thresholding in a different direction by defining an instance-dependent threshold function $c$2. Its threshold depends on instance ambiguity and instance-dependent pseudo-label error rates, using an instance-dependent transition matrix $c$3 and clean-posterior structure. That makes InstanT highly relevant as per-instance thresholding, but its adaptation is uncertainty-, ambiguity-, and noise-aware rather than semantic-aware in the feature-space or category-distribution sense emphasized by SATL (Li et al., 2023).

Older adaptive-threshold methods also illuminate the design space. ALTSS-SOM learns node-specific and dimension-specific local acceptance thresholds from local variance and feature relevance in a semi-supervised self-organizing map, making it label-aware and region-aware rather than confidence-threshold-based (Braga et al., 2019). By contrast, CLSL is closely related to SATL in application domain—multi-label image recognition with incomplete labels—and uses semantic-aware feature learning and label recovery, but it does not explicitly learn thresholds. Its refined predictions $c$4 act as semantic-aware confidence scores for missing-label recovery, which suggests a complementary rather than equivalent relation to SATL (He et al., 11 Oct 2025).

These comparisons clarify a recurring ambiguity in the literature. “Adaptive thresholding,” “instance-dependent thresholding,” and “semantic-aware thresholding” are not interchangeable. SATL is specifically category-distribution-driven threshold learning for partially labeled multi-label recognition; SST is class-aware confidence thresholding; InstanT is per-instance ambiguity-/error-aware thresholding; and CLSL is semantic-aware confidence and label-recovery modeling without explicit threshold learning.

6. Assumptions, limitations, and scope

SATL rests on a specific distributional assumption: within a category, known and unknown samples have similar score distributions. The threshold estimator therefore treats the positive and negative score distributions of known labels as proxies for the corresponding unknown-label score distributions. The paper explicitly notes that if known and unknown samples come from different domains, this assumption may fail, and threshold estimation may become unreliable (Ruan et al., 31 Jul 2025).

The method also has several presentation-level and modeling limitations. The paper repeatedly refers to “distributions,” “histograms,” and “statistical information,” but does not fully specify the practical estimator for the score distributions: exact histogram binning rules, KDE, Gaussian fitting, running-statistics formulas, and exact update granularity are not fully formalized. It states both that threshold estimation occurs during stage-two training and that “for each iteration $c$5” an ideal threshold is estimated, leaving the precise implementation granularity—strictly per mini-batch versus per epoch—under-specified (Ruan et al., 31 Jul 2025).

Another important limitation is that pseudo-label generation is one-sided. SATL defines

$c$6

but does not define an explicit negative pseudo-label rule for unknown entries. This means the method improves positive-label recovery rather than performing full three-way decision-making over positive, negative, and ignore states. A plausible implication is that its main leverage is on missing-positive recovery, not on comprehensive unknown-label disambiguation.

The term “semantic-aware” is itself narrower than it may initially appear. SATL does not inject external label text embeddings, semantic graphs, or ontological relations into threshold computation. Its semantics are category semantics manifested in score distributions. This is a meaningful but limited notion of semantic awareness. A stronger semantic-aware threshold learner would plausibly combine SATL’s category-specific distribution modeling with explicit semantic feature learning or semantic relations of the kind explored in incomplete-label multi-label methods such as CLSL (He et al., 11 Oct 2025).

Within its intended scope, however, SATL provides a precise formulation of category-specific, distribution-aware, dynamically updated threshold learning for pseudo-label mining under partial labels. Its principal contribution is to make threshold choice a learned statistical object tied to category-level score behavior and current learning state, rather than a fixed dataset-wide constant (Ruan et al., 31 Jul 2025).