Dual-Threshold Labeling Method

• Dual-threshold labeling method is a technique that uses two distinct confidence thresholds to classify predictions into high, ambiguous, and low-confidence groups.
• It enables improved calibration, selective human review, and better training by adapting thresholds based on label rarity and sample difficulty.
• The approach is applied in semi-supervised learning, hard example mining, and metric learning to enhance model robustness and overall performance.

A dual-threshold labeling method is a supervised or semi-supervised learning framework in which two distinct thresholds are used to partition instances or predictions into separate labeling or decision flows. Unlike single threshold techniques, which accept or reject predictions based on a fixed cutpoint, dual-threshold approaches distinguish between high-confidence, ambiguous, and low-confidence predictions; or, in other applications, adapt separate cutoffs for instances of differing class, label rarity, or difficulty. These methods are prevalent in semi-supervised learning, hard example mining, multi-label calibration, metric learning, and human-in-the-loop scenarios. They often motivate or enable more calibrated training, improved use of unlabeled or minority-class data, better trade-offs in operational pipelines, and robust transfer under distribution shift.

1. Formal Definitions and Architectures

The dual-threshold paradigm appears in several forms:

• Fixed Upper/Lower Band: Instances are partitioned using two thresholds $(\tau_\mathrm{low}, \tau_\mathrm{high})$ on a probability or confidence score $p$. Typical regions are:
  • $p < \tau_\mathrm{low}$: auto-label negative (or class 0)
  • $\tau_\mathrm{low} \leq p < \tau_\mathrm{high}$: ambiguous/review region (defer, human, or extra processing)
  • $p \geq \tau_\mathrm{high}$: auto-label positive (or class 1)
  • This structure underpins human-in-the-loop frameworks and hard example mining (Muric et al., 9 Jan 2026, Ren et al., 26 Jun 2026).
• Class-Adaptive or Label-Specific Bands: A fixed threshold is supplemented with dynamic or class-adaptive thresholds. In ADT-SSL, a high fixed threshold $\tau_f$ (e.g., 0.95) is paired with a class-specific adaptive $\tau_c$ determined from the distribution of labeled confidences (Liang et al., 2022):
  • $p \geq \tau_f$: “easy” high-confidence pseudo-labels.
  • $\tau_c \leq p < \tau_f$: “hard” samples still above the per-class minimum.
• Fusion of Global and Local Statistics: The “dual-threshold” in multi-label tasks can refer to the learned fusion of global label rarity (IDF) and batch-local KNN co-occurrence to set per-label, per-instance activation bars (Shamatrin, 6 May 2025):

$$\tau_{i,\ell} = \lambda \alpha_\ell g_\ell + (1-\lambda) \beta_\ell \ell_{i,\ell} + b_\ell$$

where $p$0 is IDF-style rarity and $p$1 is batch-local agreement.

• Sample Mining with Dual Margins: In deep metric learning, two margins—one for positive pairs ($p$2) and one for negatives ($p$3)—define two thresholds for mining informative pairs (Jiang et al., 2024).

2. Algorithmic Implementations

Implementations vary by application but share the structure of dynamic or static assignment of samples to mutually exclusive decision or training roles based on threshold crossing.

• Human-Machine Pipelines: For a calibrated classifier outputting $p$4, two thresholds $p$5 partition predictions as follows:
  • $p$6: auto-negative.
  • $p$7: auto-positive.
  • $p$8: escalate to human review.
  • Optimizing these boundaries can be formalized as a Lagrangian or constrained budget problem, trading off accuracy against review rate. Under 0–1 loss and perfectly calibrated probabilities, the analytically optimal policy is symmetric: $p$9, $p < \tau_\mathrm{low}$0 for a penalty parameter $p < \tau_\mathrm{low}$1 (Muric et al., 9 Jan 2026).
• Semi-Supervised Learning (ADT-SSL): The fixed threshold $p < \tau_\mathrm{low}$2 selects “easy” pseudo-labeled unlabeled examples (high-confidence), while class-adaptive $p < \tau_\mathrm{low}$3 enables inclusion of “hard” cases. CE loss is applied to $p < \tau_\mathrm{low}$4 (above $p < \tau_\mathrm{low}$5), $p < \tau_\mathrm{low}$6 consistency to $p < \tau_\mathrm{low}$7 (between $p < \tau_\mathrm{low}$8 and $p < \tau_\mathrm{low}$9), with a third “similar pairs” loss to maintain consistency in highly similar samples (Liang et al., 2022).
• Hard Example Mining: In adversarial or domain-transfer settings, like cross-platform offensive comment detection, two empirically selected confidence cutoffs (e.g., $\tau_\mathrm{low} \leq p < \tau_\mathrm{high}$0, $\tau_\mathrm{low} \leq p < \tau_\mathrm{high}$1) identify high-confidence false negatives and low-confidence ambiguous cases. These are prioritized for manual labeling and incremental fine-tuning (Ren et al., 26 Jun 2026).
• Label-Specific Adaptive Thresholds in Multi-Label Learning: IDF-KNN fusion adaptively modulates activation for each label-instance pair, shifting loss penalties and margins, rather than applying hard cut-offs. This yields a smooth, differentiable realization of the dual-threshold concept (Shamatrin, 6 May 2025).
• Pairwise Metric Learning: Static dual margins are set for positive and negative pairs, while adaptive tolerance mechanisms (AT-ASMS) dynamically adjust these to maintain a balanced mined pair ratio, using a meta-learning gradient for updating the loss margin (Jiang et al., 2024).

3. Applications and Empirical Results

Dual-threshold techniques have demonstrated empirical utility across diverse tasks:

• Human-in-the-Loop Classification: Monte Carlo simulations show that double-threshold policies yield a sharply increasing accuracy for small human-review budgets, followed by a plateau of diminishing returns—quantitatively enabling selection of review rates aligned with operational capacity (Muric et al., 9 Jan 2026).
• Semi-Supervised Vision Classification: On CIFAR-10 with 4000 labels (8% labeled), ADT-SSL matches fully supervised performance (95.74% vs 95.75%). On CIFAR-100, replacing the single threshold with dual (fixed + adaptive per-class) adds 0.56%–3.15% accuracy; ablation studies confirm each component contributes additively (Liang et al., 2022).
• Domain Adaptation and Hard Mining: For Chinese offensive comment detection, fine-tuning on just 400 dual-threshold-mined hard examples boosts macro-F1 by $\tau_\mathrm{low} \leq p < \tau_\mathrm{high}$2 to $\tau_\mathrm{low} \leq p < \tau_\mathrm{high}$3 over baseline across four social media platforms, while random sampling degrades performance (Ren et al., 26 Jun 2026).
• Multi-Label Long-Tailed Regimes: On AmazonCat-13K, dual-threshold (IDF+KNN) fusion attains macro-F1 0.1712, outperforming prior bests (AttentionXML $\tau_\mathrm{low} \leq p < \tau_\mathrm{high}$4 0.07) and static baselines (macro-F1$\tau_\mathrm{low} \leq p < \tau_\mathrm{high}$50.01 for “0.5” threshold) (Shamatrin, 6 May 2025).
• Deep Metric Learning: The Dual Dynamic Threshold Adjustment Strategy maintains a balanced positive:negative pair ratio throughout training, enabling +2.7% recall@1 on CUB200 and +2.3% on Cars196 versus strong baselines (Jiang et al., 2024).

4. Design Principles and Threshold Calibration

Common calibration methods:

• Grid Search on Error Histograms: For domain adaptation, thresholds are selected via grid search to balance the number and difficulty of error-prone samples (e.g., maximizing hard examples while avoiding ambiguous, mid-confidence zones) (Ren et al., 26 Jun 2026).
• Online Class-Adaptive Estimation: In ADT-SSL, per-class adaptive thresholds $\tau_\mathrm{low} \leq p < \tau_\mathrm{high}$6 are updated online from labeled data by tracking the minimum confidence of correctly classified instances per class. These are reset each epoch to prevent premature over-tightening (Liang et al., 2022).
• Meta-Learning for Margin Adaptation: In pairwise metric settings, a meta-learning, one-step gradient approach updates the loss threshold based on validation meta-batches, enabling rapid responsiveness to evolving sample distributions (Jiang et al., 2024).
• Learnable Fusion Parameters: For global-local threshold fusion, the weights $\tau_\mathrm{low} \leq p < \tau_\mathrm{high}$7 and biases $\tau_\mathrm{low} \leq p < \tau_\mathrm{high}$8 are trained end-to-end, automatically balancing global rarity against local co-labeling (Shamatrin, 6 May 2025).

5. Loss Functions and Training Objectives

Typical loss constructs:

• Cross-Entropy (CE): Employed for definitive or high-confidence samples.
• Consistency Regularization (L2 Loss): For samples between thresholds, as in ADT-SSL.
• Margin/Sample Mining Loss: For metric learning, separate margins for pos/neg pairs yield asymmetric penalties.
• Shifted-Logit Binary Cross-Entropy: Multi-label fusion strategies subtract the threshold before BCE; a soft margin penalty further sharpens decision boundaries (Shamatrin, 6 May 2025).
• Global Loss Aggregation: Aggregated loss combines primary (labeled), multiple unsupervised, and auxiliary (e.g., similar-pair) objectives, weighted per-dataset (Liang et al., 2022).

6. Interpretability, Modularity, and Adaptability

Dual-threshold methods are interpretable:

• Decision-Band Transparency: The trigger points and decision regions are explicit; for instance, IDF-KNN weights directly encode how much influence global rarity or local affinity exerts (Shamatrin, 6 May 2025).
• Modularity: Thresholding heads or hard mining modules can be appended atop arbitrary feature backbones, including transformer encoders and convolutional nets, without retraining the feature extractor (Shamatrin, 6 May 2025).
• Dynamic Control: Threshold values and bands can be periodically recomputed to respond to drift (e.g., class prevalence) or operational changes (e.g., human review bandwidth) (Muric et al., 9 Jan 2026).

7. Limitations and Future Directions

Notable caveats include:

• Reviewer Imperfection: Human review is rarely perfect; the dual-threshold theory assumes perfect accuracy in the escalation band—extensions accommodate reviewer error by adjusting marginal gain equations (Muric et al., 9 Jan 2026).
• Parameter Sensitivity: Plain random sampling or static choices for threshold placement may be suboptimal or even detrimental, as shown empirically in hard example mining for domain adaptation (Ren et al., 26 Jun 2026).
• Extension to Multi-Class and Cost-Sensitive Settings: Generalization requires per-class threshold bands or cost-calibrated utility, increasing both complexity and the need for robust calibration (Muric et al., 9 Jan 2026, Liang et al., 2022).
• Integration with Graph-Based and Task-Conditional Thresholding: Future work may replace local KNN with learned graphs, fuse TF-IDF-style features, or embed thresholding deeper in network architectures (Shamatrin, 6 May 2025).

In summary, dual-threshold labeling methods operationalize a principled, flexible approach for partitioning predictions or examples, enabling selective training, calibration, or intervention. Applications span semi-supervised learning, multi-label and metric learning, hard example domain adaptation, and human-in-the-loop AI, with consistent evidence of improved calibration, robustness, and performance across domains (Liang et al., 2022, Shamatrin, 6 May 2025, Jiang et al., 2024, Ren et al., 26 Jun 2026, Muric et al., 9 Jan 2026).