Dynamic Threshold Adjustment (DTA)

• Dynamic Threshold Adjustment is a method that adaptively modulates decision thresholds during training to manage label ambiguity and class imbalance.
• It is implemented through techniques like threshold-based distribution correction, variance-driven attention partitioning, and sample-adaptive fusion, ensuring robust performance in facial expression recognition.
• DTA mitigates issues from noisy and sparse data by dynamically adjusting per-class and per-sample weights, resulting in improved model generalization and fairness.

Dynamic Threshold Adjustment (DTA) refers to the class of methodologies in machine learning, primarily for imbalanced or noisy supervised learning tasks, that adapt class or decision thresholds during training or inference in response to empirical statistics, label ambiguities, or feature-space uncertainties. DTA is particularly prevalent in deep facial expression recognition (FER), where challenges of annotation ambiguity, label distribution imbalance, and subtle inter-class separability demand robust sample- or class-level adaptation. In recent advances, DTA has been instantiated as explicit per-class thresholding in label-distribution learning (Liu et al., 2024), implicit regularization via dynamic partitioning of attention heads (Wen et al., 2021), and conditional replacement of unstable distribution estimates.

1. Conceptual Overview and Motivation

DTA targets key weaknesses in one-hot or fixed-threshold supervised learning: failure to accommodate ambiguous labels, sample-level heterogeneity, and class imbalance. In FER, annotator disagreement and rare expressions (such as "Disgust" or "Fear"), lead to fluctuation in both the empirical and modeled label distributions. Static decision boundaries fail to capture these subtleties, resulting in performance degradation.

Adaptive thresholding dynamically modulates the influence of individual samples or classes. Typical motivations include:

• Mitigating label noise by upweighting high-confidence samples.
• Enforcing class priors or distribution balance.
• Guarding against overfitting on unstable or ambiguous regions.
• Improving generalization through dynamically fused supervision signals.

2. DTA Instantiations in Deep Learning Architectures

Recent FER works implement DTA in several mathematically grounded ways:

a. Threshold-Based Distribution Correction

"Ada-DF: An Adaptive Label Distribution Fusion Network For Facial Expression Recognition" (Liu et al., 2024) embeds DTA directly within its class distribution estimation. Consider the per-class distribution $d_{class}^c$, computed as the mean of per-sample auxiliary outputs. Empirically, $d_{class}^{c,c}$ (the probability assigned by class $c$ to itself) may fall below a threshold $t$ due to label ambiguity or class rarity. The Ada-DF framework then substitutes $d_{class}^c$ with a thresholded distribution: $d_{thre}^{c, y_j} = \begin{cases} t & \text{if } j = c \[4pt] \frac{1-t}{C-1} & \text{otherwise} \end{cases}$ This substitution occurs when $d_{class}^{c,c} < t$, effectively preventing the auxiliary branch from generating degenerate or misleading soft labels in early training phases.

b. Diversity-Promoting Loss with Partition Variance

Within the Distract Your Attention Network (DAN) (Wen et al., 2021), thresholding manifests as a variance-driven penalty in the attention fusion stage. The partition loss

$$L_{pt} = \frac{1}{N L} \sum_{i=1}^{N} \sum_{l=1}^{L} \log\Big(1 + \frac{K}{\sigma^2_{i,l}}\Big)$$

where $\sigma^2_{i,l}$ is the across-head variance per feature-dimension, contextually enforces separation among feature subspaces attended to by different heads. Regions of low variance are penalized, guaranteeing "distracted" (non-overlapping) attention. While not a threshold in the classifier output itself, it is a dynamic adjustment of feature-space allocations—effectively enforcing partitioned response below an adaptive variance level.

c. Sample-Adaptive Fusion via Attention

Ada-DF (Liu et al., 2024) further adapts the fusion weights of per-sample and per-class distributions through normalized attention scores. For each sample, weights $w_{aux, x_i}$, $w_{tar, x_i}$ are computed and normalized so that no sample fully ignores any distribution. The fusion

$$d_{fused, x_i} = w_{x_i} d_{class}^{y_i} + (1 - w_{x_i}) d_{label, x_i}$$

is adaptive, modulating thresholding on a per-instance basis depending on empirical confidence.

3. Mathematical Formalism and Training Protocols

DTA mechanisms appear in supervised losses via:

• Explicit per-class threshold logic (see the threshold distribution above).
• Rank regularization terms that maintain separation between high- and low-confidence samples: $L_{RR} = \max(0, \delta - (w_H - w_L))$ with $w_H$ as mean top-$M$ attention weights and $w_L$ as mean remaining batch weights (Liu et al., 2024).
• Weighted cross-entropy losses in uncertain label correction, where per-sample confidence $\alpha_i$ and per-batch class reweighting $\gamma_j$ adjust the influence of samples relative to batch-level distributions (Liu et al., 2022).

The full training objectives combine categorical (softmax or KL divergence) losses with these dynamic adjustment penalties, anchoring gradient updates to current statistics of the batch or the sample label distribution.

4. Empirical Impact and Benchmark Results

Evaluations across RAF-DB and AffectNet FER benchmarks reveal that DTA-equipped architectures outperform static-threshold models, CNNs, and vanilla transformer approaches:

Model	DTA Method	RAF-DB Acc.	AffectNet Acc.
Ada-DF (Liu et al., 2024)	Thresholded LDL Fusion	–	–
DAN (Wen et al., 2021)	Partition Loss + Affinity	89.70%	62.09–65.69%
ULC-AG (Liu et al., 2022)	Confidence Weighting	89.31%	61.57%

These results demonstrate that DTA not only stabilizes training in the presence of ambiguous annotations but also enables networks to specialize in rare classes and ambiguous samples, closing the gap with state-of-the-art models even in challenging, noisy data regimes.

5. Relationships to Label Distribution Learning and Uncertainty Modeling

DTA is tightly coupled with label distribution learning (LDL), where the strict discrete target per sample is replaced by a probabilistic vector. Threshold replacement, as in Ada-DF, directly modulates LDL by substituting insufficiently confident class descriptions. Weighted regularization, as in ULC-AG (Liu et al., 2022), dynamically absorbs sample label uncertainty into the learning objective.

DTA also relates to uncertainty-aware modeling, where confidences or reliability scores govern the degree to which samples participate in parameter updates. Dynamic adjustment mechanisms thus intersect with broader topics in robust learning, especially in the context of noisy and imbalanced data prevalent in large-scale real-world datasets.

6. Considerations in Dataset Design and Model Evaluation

Dynamic thresholding is effective when dataset composition or annotation reliability is variable. Studies of RAF-DB and AffectNet show extreme class imbalance (AffectNet IR ≈ 21, RAF-DB IR ≈ 4) (Hosseini et al., 16 Feb 2025), with significant demographic and annotator disagreement biases. DTA-equipped models are less sensitive to these artifacts, leveraging adaptive logic to smooth their supervision and avoid overfitting to dominant patterns.

DTA also interacts with fairness: adaptive thresholding may benefit under-represented classes, though bias and parity measures (demographic parity difference, equalized odds) must be tracked to ensure that dynamic logic does not propagate or exacerbate demographic disparities in model outputs.

7. Controversies and Future Directions

While DTA has achieved measurable gains, debates persist regarding optimality of threshold selection (fixed vs. learned), stability across epochs, and scaling of adjustment mechanisms. Some researchers advocate for end-to-end learnable thresholds based on principled uncertainty metrics; others prefer fixed policy-driven logic to prevent drift and mode collapse in extremely class-skewed regimes.

Advances in self-supervised pretraining and generative data augmentation may further mitigate the need for aggressive DTA, but for highly ambiguous data, dynamic threshold adjustment remains a cornerstone of modern FER pipelines (Liu et al., 2024, Wen et al., 2021). A plausible implication is that as label ambiguity and real-world scenario diversity increase, DTA paradigms will become foundational in interpretable and robust affective computing.