---
title: Uncertainty-aware Dynamic Threshold Selection (UDTS)
url: https://www.emergentmind.com/topics/uncertainty-aware-dynamic-threshold-selection-udts
type: topic
---

# Uncertainty-aware Dynamic Threshold Selection (UDTS)

Uncertainty-aware Dynamic Threshold Selection (UDTS) is a general mathematical and algorithmic framework that adaptively selects thresholds for decision-making or statistical inference in the presence of aleatoric and epistemic uncertainty. Broadly, UDTS encompasses methods in statistical inference, machine learning, combinatorial optimization, and sequential decision processes where the threshold is not predetermined but is dynamically selected based on estimated uncertainty, aiming to optimize prediction quality, coverage guarantees, robustness, or sample efficiency. Applications span from extreme value analysis and robust classification to semi-supervised learning and real-time autonomous systems.

## 1. Theoretical Foundations and Core Principles

Central to UDTS is the interdependence between threshold selection and uncertainty quantification. In traditional threshold-based methodologies—such as generalized Pareto (GP) modeling, conformal prediction, or combinatorial set selection—the threshold defines the subset of data or solution space that controls the trade-off between bias and variance, prediction region volume, or inference stability. UDTS extends this by incorporating:

- Formal uncertainty estimation for the predictive modeling or the decision variable (entropy, variance, nonconformity scores, evidence mass, or optimality intervals);
- Dynamic or sequential adjustment of the threshold as a function of observed uncertainty;
- Explicit propagation of threshold uncertainty into downstream predictions.

These principles address the classical dilemma where static thresholds either fail under distributional shift, adversarial perturbations, or class imbalance, or require extensive pre-tuning.

## 2. Methodologies and Algorithmic Realizations

Multiple methodological instantiations of UDTS exist:

### 2.1. Bayesian and Bootstrap-Based UDTS for Extremes

In extreme value statistics, UDTS is realized via empirical loss or risk minimization over candidate thresholds, using discrepancy measures that quantify the fit between model and observed quantiles. For instance, the method of "Automated threshold selection and associated inference uncertainty for univariate extremes" [2310.17999] minimizes an integrated absolute error metric over grid-searches of possible thresholds and propagates threshold uncertainty via bootstrap resampling to confidence intervals for return levels. Bayesian cross-validation and model-averaging further refine the approach by averaging predictions over thresholds weighted by their posterior predictive risk, as in [1504.06653].

### 2.2. Uncertainty-Aware Thresholds in Deep Learning

In supervised and semi-supervised learning, UDTS forms the basis for selection of pseudo-labels or ensemble members. Examples include:

- Epistemic uncertainty estimation via Monte Carlo dropout, followed by per-class, exponentially-smoothed thresholds for pseudo-label selection in long-tailed datasets [2401.04435]. Selection criteria combine both model confidence and uncertainty, with thresholds dynamically adapted to learning stage and class frequency.
- Dynamic model selection in deep neural network ensembles for adversarial robustness, where sub-models output Dirichlet concentration parameters, uncertainty is quantified using Dirichlet entropy, and the sub-model with minimum uncertainty is greedily selected for prediction. No aggregation or voting is used—selection is hard and uncertainty-driven [2308.00346].

### 2.3. Online and Sequential UDTS in Conformal Prediction

Real-time prediction under non-exchangeable or drifting distributions (e.g., in robotics) leverages UDTS via online adaptation of both the nonconformity score function and the quantile threshold. "AdaptNC" [2602.01629] combines stochastic approximation of quantile thresholds (DtACI) with a replay buffer for coverage stability and periodic adaptation of score functions to maintain target coverage and shrinkage of prediction region volume.

### 2.4. Combinatorial Optimization under Unstable Weights

In combinatorial optimization, UDTS formalizes the process of maintaining inclusion and exclusion thresholds for each element under uncertainty intervals. For each element, two thresholds (t_e^–, t_e^+) demarcate where it is always in or always out of all optima, allowing for efficient minimum admissible querying and dynamic adaptation as weights are revealed sequentially [2404.17214].

## 3. Representative Algorithms and Mathematical Statements

UDTS is instantiated with task-specific but structurally similar selection rules. The following table summarizes representative algorithmic patterns:

| Task/Domain            | Uncertainty Metric                    | Dynamic Selection Rule                                           |
|------------------------|--------------------------------------|-----------------------------------------------------------------|
| Extreme Value Analysis | Empirical quantile discrepancy (boot.)| Select threshold u* = argmin_u estimated risk                   |
| Ensemble DNN           | Dirichlet entropy                     | Select m* = argmin_m H^{(m)}(x); output model m*                |
| Semi-SL                | MC Dropout entropy                    | Select x with u(x) ≤ T_c^(t), p_c(x) ≥ T_c^(t)                  |
| Conformal Prediction   | Nonconformity score                   | Tune τ_t online for target coverage, adapt score shape θ_t       |
| Set Selection          | Inclusion/exclusion thresholds        | Update (t_e^–, t_e^+) as intervals contract/revealings occur     |

All realizations rely on updating thresholds in response to uncertainty as measured either by explicit predictive entropy, nonconformity statistics, or changes in the value function under parametric uncertainty.

## 4. Empirical Performance and Application Case Studies

Application-specific empirical findings include:

- In multi-omics decision systems, UDTS enables over 60% of patients to be classified confidently using a single data modality, matching the accuracy of full-modal integration while drastically reducing data acquisition costs [2507.01032].
- In long-tailed semi-supervised image classification, UDTS yields consistent improvements in accuracy relative to static-threshold FixMatch baselines, with gains of 1–10% absolute depending on dataset and experimental settings [2401.04435].
- In online conformal prediction for robotic systems, AdaptNC (a UDTS instance) achieves target coverage with region volumes reduced by 30–90% compared to fixed-threshold/score methods, and avoids instability in coverage under sudden environment shifts [2602.01629].
- In threshold selection for ocean storm extremes, Bayesian model-averaged UDTS reduces sensitivity to threshold choice and achieves robustness with respect to return-level estimation uncertainty [1504.06653].

## 5. Sensitivity, Uncertainty Propagation, and Limitations

A defining characteristic of UDTS is explicit propagation of threshold uncertainty to downstream statistical inference or decisions. In statistical extremes, this is realized via double bootstrap variance estimation for return-level confidence intervals; in combinatorial optimization, ambiguity about membership in any optimum induces further querying/adaptation. All current frameworks assume access to either an accurate uncertainty measure or efficient computation of inclusion/exclusion thresholds. Challenges exist in highly non-stationary, high-dimensional, or non-convex domains:

- For prediction under dependency or non-iid settings, threshold weights may need continual adaptation and buffer-based recalibration.
- Some optimality threshold problems (e.g., shortest s–t path edge inclusion/exclusion under interval uncertainty) are NP-hard, limiting UDTS tractability in these classes [2404.17214].
- Empirical threshold selection remains sensitive to choice of discrepancy measure, smoothing hyperparameters, and window sizes in dynamic regimes.

## 6. Relations to Broader Research and Extensions

UDTS unifies a class of adaptive selection strategies motivated by uncertainty quantification, providing a rigorous mathematical scaffold applicable across statistics, learning, decision theory, and combinatorial structures. It is closely related to dynamic risk minimization, online learning with partial information, and robust inference under model misspecification. Ongoing research includes extension to:

- Multi-view and multi-modal data with complex fusion rules (e.g., Dempster–Shafer based evidence accumulation [2507.01032]);
- Non-iid and adversarial domains using replay or buffer stabilization [2602.01629];
- Integration with structured covariate models and nonparametric density estimation for time-varying thresholding in extremes [2310.17999].

A plausible implication is that as predictive modeling transitions to high-stakes applications with distributional uncertainty and heterogeneity, UDTS-like procedures will form the backbone of robust, adaptive, and cost-efficient decision-making frameworks.

Source: https://www.emergentmind.com/topics/uncertainty-aware-dynamic-threshold-selection-udts