Scale-Aware Thresholding

• Scale-aware thresholding is a principle where thresholds adapt dynamically based on observed scale features such as mean, variance, or group size, improving control over detection and sparsity.
• It finds applications in anomaly detection, kernel density estimation, sparse optimization, and network analysis, tailoring thresholds to context-specific scale variables.
• Empirical studies demonstrate that scale-sensitive methods achieve significant performance gains and reveal limitations of fixed thresholds, prompting more adaptive designs.

Scale-aware thresholding denotes a family of thresholding formulations in which the threshold, the thresholding rule, or the interpretation of thresholded outcomes is made explicitly dependent on a scale variable. In the surveyed literature, that scale variable may be the recent mean and variance of anomaly scores, the bandwidth of a kernel density estimate, the norm magnitude of a coefficient group, the empirical distribution of layer activations, the variance attached to each test statistic, or the size parameter implicitly imposed by thresholding itself. The common theme is that thresholding is not treated as a fixed post-processing constant, but as a scale-sensitive mechanism for controlling detection, sparsity, segmentation, inference, or graph renormalization (Yang et al., 2023).

1. Conceptual foundations and recurring mathematical patterns

A first axis of the subject concerns thresholds that adapt to an observed scale. In anomaly detection, ADT encodes the recent anomaly-score mean and variance in the RL state, namely

$$s_t = \{\mu_t, \sigma_t, \rho_t^{TP}, \rho_t^{TN}, \rho_t^{FP}, \rho_t^{FN}\},$$

so that threshold selection reacts to shifts in the central tendency and dispersion of scores (Yang et al., 2023). In histogram thresholding, the relevant scale is the KDE bandwidth $\sigma^2$, and the method searches for the scale at which the number of minima equals the requested number of classes minus one (Korneev et al., 2022). In sparse optimization, FITS$^3$ uses group-specific thresholds proportional to $\psi'(\|\tilde{x}_g^k\|_p)$, so smaller groups are penalized more aggressively than larger ones (Zhao et al., 8 Jan 2025). In LLM sparsification, CATS calibrates a per-layer threshold

$\tau_l = F_l^{-1}(k)$

from the empirical CDF of absolute gate activations, making the threshold dependent on layer-wise activation scale (Lee et al., 2024). In multiple testing, HART uses the pair $(X_i,\sigma_i)$ rather than a standardized statistic alone, via the conditional local false discovery rate

$T_i = \mathbb{P}(\theta_i=0\mid x_i,\sigma_i),$

so that ranking and thresholding remain variance-aware (Fu et al., 2019).

A second axis concerns thresholds that themselves impose a scale. The birth–death-process analysis shows that thresholding can generate a threshold-imposed crossover scale,

$S_\theta = 8\pi h,$

which separates a threshold-induced regime from the true asymptotic tail (Font-Clos et al., 2014). In hidden-variable network models, the threshold parameter must scale with network size, such as $T(N)\sim \frac{1}{\lambda}\ln N$ in the classical exponential hard-threshold model, in order to preserve sparsity while maintaining $p(k)\sim k^{-2}$ (Balogh et al., 2019). Degree-thresholding renormalization similarly uses the retained relative network size $\sigma^2$0 as the relevant scale variable for finite-size scaling (Chen et al., 2021).

A third axis concerns bridging between hard and soft thresholding by explicit scaling. Scaled soft thresholding expands soft-thresholded coefficients by empirical scaling values indexed by an order parameter $\sigma^2$1, with soft thresholding at $\sigma^2$2, the non-negative garrote at $\sigma^2$3, and convergence toward hard thresholding as $\sigma^2$4 increases (Hagiwara, 2021). Adaptive scaling for soft-thresholding in orthogonal regression uses component-wise factors

$\sigma^2$5

for selected coefficients, so that threshold level and shrinkage are controlled separately (Hagiwara, 2016).

These formulations suggest that “scale-aware thresholding” is not a single algorithmic template. Rather, it is a design principle that recurs whenever a threshold must be stabilized against heterogeneity, non-stationarity, overlap, or finite-size effects.

2. Dynamic threshold control in anomaly detection

In "ADT: Agent-based Dynamic Thresholding for Anomaly Detection" (Yang et al., 2023), thresholding is modeled as a Markov Decision Process. A multivariate time series $\sigma^2$6 is normalized to $\sigma^2$7 and partitioned into sliding windows of length $\sigma^2$8 with stride $\sigma^2$9, producing windows $^3$0. An autoencoder supplies anomaly scores $^3$1, and ADT uses a Deep Q-Network to choose the threshold action in real time.

The scale-aware part of ADT lies in the state:

$^3$2

where $^3$3 and $^3$4 summarize the recent anomaly-score distribution over a lookback horizon $^3$5, and the outcome proportions summarize recent classifier behavior under the current threshold mode. The reward is

$^3$6

with $^3$7 and $^3$8, allowing explicit control of aggressiveness versus conservativeness (Yang et al., 2023).

ADT uses a binary threshold action $^3$9 with active mode $\psi'(\|\tilde{x}_g^k\|_p)$0 and passive mode $\psi'(\|\tilde{x}_g^k\|_p)$1, and the classification rule

$\psi'(\|\tilde{x}_g^k\|_p)$2

The DQN objective is based on the standard state-action value

$\psi'(\|\tilde{x}_g^k\|_p)$3

with replay memory, a target network, and $\psi'(\|\tilde{x}_g^k\|_p)$4-greedy exploration. The paper reports that actions are updated every $\psi'(\|\tilde{x}_g^k\|_p)$5 steps during training and that the Q-network is updated only at the end of each episode, with small training sets, specifically “<1% of data,” used to demonstrate data-efficient learning (Yang et al., 2023).

The empirical results are unusually strong. On Yahoo, SWaT, and WADI, ADT achieved P/R/F1 of $\psi'(\|\tilde{x}_g^k\|_p)$6, $\psi'(\|\tilde{x}_g^k\|_p)$7, and $\psi'(\|\tilde{x}_g^k\|_p)$8, respectively; the corresponding Static and DSPOT baselines were substantially lower in F1 on all three datasets (Yang et al., 2023). The feasibility study on a SWaT subsequence reported that ADT flipped thresholds to $\psi'(\|\tilde{x}_g^k\|_p)$9 in anomalous runs and $\tau_l = F_l^{-1}(k)$0 in normal runs, with classification accuracy $\tau_l = F_l^{-1}(k)$1 (Yang et al., 2023).

Within this formulation, scale awareness is operational rather than merely descriptive. Because $\tau_l = F_l^{-1}(k)$2 and $\tau_l = F_l^{-1}(k)$3 enter the state directly, the controller reacts to changes in anomaly-score scale and dispersion rather than relying on a fixed absolute cutoff. The paper also notes a limitation: the reward requires access to labeled windows during training, and the controller only chooses $\tau_l = F_l^{-1}(k)$4, which may be coarse when anomaly-score distributions overlap heavily (Yang et al., 2023).

3. Scale-space thresholding and threshold-imposed scales

A distinct line of work uses scale in the sense of smoothing scale. The KDE-based multiclass histogram thresholding method models a histogram as

$\tau_l = F_l^{-1}(k)$5

with Gaussian kernels centered at histogram bin locations and unknown weights estimated by an EM-type multiplicative deconvolution equivalent to the Richardson–Lucy algorithm (Korneev et al., 2022). Thresholds are the local minima of the KDE, characterized in one dimension by $\tau_l = F_l^{-1}(k)$6 and $\tau_l = F_l^{-1}(k)$7.

The key scale-aware mechanism is the monotonic search over bandwidth. Gaussian scale-space guarantees non-enhancement of local extrema: as $\tau_l = F_l^{-1}(k)$8 increases, no new extrema are created, so the number of minima decreases monotonically. The method therefore brackets a bandwidth interval and performs binary search until the number of minima matches $\tau_l = F_l^{-1}(k)$9, then chooses the smallest $(X_i,\sigma_i)$0 that yields that count (Korneev et al., 2022). The reported total complexity is

$(X_i,\sigma_i)$1

with FFT-based convolutions throughout (Korneev et al., 2022).

The empirical record in this paper is domain-specific but concrete. On $(X_i,\sigma_i)$2 synthetic histograms from mixtures of three Cauchy components, deviations above $(X_i,\sigma_i)$3 intensity unit occurred in about $(X_i,\sigma_i)$4 of predictions with $(X_i,\sigma_i)$5 bins and were reduced to approximately $(X_i,\sigma_i)$6 with $(X_i,\sigma_i)$7 bins (Korneev et al., 2022). In the XCT porosity application, the estimated porosity was $(X_i,\sigma_i)$8 against an experimental measurement of $(X_i,\sigma_i)$9, a relative error of $T_i = \mathbb{P}(\theta_i=0\mid x_i,\sigma_i),$0, whereas K-means segmentation yielded porosity $T_i = \mathbb{P}(\theta_i=0\mid x_i,\sigma_i),$1 with $T_i = \mathbb{P}(\theta_i=0\mid x_i,\sigma_i),$2 error (Korneev et al., 2022).

A different notion of scale appears in "The perils of thresholding" (Font-Clos et al., 2014), where thresholding is analyzed not as adaptation to scale but as a source of scale. For the thresholded birth–death process, the event-duration distribution exhibits three regimes:

$T_i = \mathbb{P}(\theta_i=0\mid x_i,\sigma_i),$3

$T_i = \mathbb{P}(\theta_i=0\mid x_i,\sigma_i),$4

and

$T_i = \mathbb{P}(\theta_i=0\mid x_i,\sigma_i),$5

The threshold therefore introduces a spurious scaling region with exponent $T_i = \mathbb{P}(\theta_i=0\mid x_i,\sigma_i),$6 and a crossover scale

$T_i = \mathbb{P}(\theta_i=0\mid x_i,\sigma_i),$7

while the true asymptotic exponent remains $T_i = \mathbb{P}(\theta_i=0\mid x_i,\sigma_i),$8 (Font-Clos et al., 2014).

The paper’s proposed diagnostic is a threshold-aware data collapse. Plotting $T_i = \mathbb{P}(\theta_i=0\mid x_i,\sigma_i),$9 versus $S_\theta = 8\pi h,$0 exposes the imposed scale and the crossover near $S_\theta = 8\pi h,$1, while a complementary collapse isolates the threshold-dominated regime (Font-Clos et al., 2014). This is an important corrective to a common misconception: thresholding does not merely reveal existing scale-free structure; it can create apparent scaling exponents that are artifacts of the threshold.

4. Scale-aware sparsification and sparse estimation

In optimization and signal recovery, scale-aware thresholding is often realized through adaptive shrinkage magnitude. FITS$S_\theta = 8\pi h,$2 solves

$S_\theta = 8\pi h,$3

with nonconvex non-Lipschitz group regularization (Zhao et al., 8 Jan 2025). The method combines hard pruning of groups below a user-chosen threshold $S_\theta = 8\pi h,$4 with groupwise shrinkage using thresholds proportional to $S_\theta = 8\pi h,$5. For $S_\theta = 8\pi h,$6, the update is block soft-thresholding:

$S_\theta = 8\pi h,$7

Because $S_\theta = 8\pi h,$8 and decreases with group norm, small-scale groups receive larger thresholds and large-scale groups are preserved more readily (Zhao et al., 8 Jan 2025).

The paper gives both a lower-bound theory and convergence guarantees under KŁ assumptions, and reports that FITS$S_\theta = 8\pi h,$9 “costs only around a half of the CPU time by the second fastest comparator” on median or large-scale problems (Zhao et al., 8 Jan 2025). At $T(N)\sim \frac{1}{\lambda}\ln N$0 and sparsity level $T(N)\sim \frac{1}{\lambda}\ln N$1, FITS$T(N)\sim \frac{1}{\lambda}\ln N$2 required $T(N)\sim \frac{1}{\lambda}\ln N$3s with $T(N)\sim \frac{1}{\lambda}\ln N$4, compared with $T(N)\sim \frac{1}{\lambda}\ln N$5s for IRLS-th, $T(N)\sim \frac{1}{\lambda}\ln N$6s for InISSAPL-GSO, $T(N)\sim \frac{1}{\lambda}\ln N$7s for PGM-GSO, and $T(N)\sim \frac{1}{\lambda}\ln N$8s for ADMM-GL (Zhao et al., 8 Jan 2025).

In orthogonal regression and wavelet denoising, two related papers formulate scale awareness by scaling soft-thresholded coefficients. "Bridging between soft and hard thresholding by scaling" defines a scaled soft-thresholding estimator

$T(N)\sim \frac{1}{\lambda}\ln N$9

for $p(k)\sim k^{-2}$0, where

$p(k)\sim k^{-2}$1

Here $p(k)\sim k^{-2}$2 recovers soft thresholding, $p(k)\sim k^{-2}$3 yields the non-negative garrote, and large $p(k)\sim k^{-2}$4 approaches hard thresholding (Hagiwara, 2021). The degrees of freedom decompose as

$p(k)\sim k^{-2}$5

which the paper interprets as exposing two sources of over-fitting: active-set size and a remainder term connecting to hard-threshold search degrees of freedom (Hagiwara, 2021).

"Adaptive scaling for soft-thresholding estimator" introduces component-wise data-dependent factors

$p(k)\sim k^{-2}$6

after LARS-based soft-thresholding (Hagiwara, 2016). The associated unbiased risk estimate is

$p(k)\sim k^{-2}$7

In the orthogonal regression simulation, adaptive scaling yielded risk $p(k)\sim k^{-2}$8 with $p(k)\sim k^{-2}$9 selected components on average, compared with $\sigma^2$00 and $\sigma^2$01 for LST and $\sigma^2$02 and $\sigma^2$03 for LST-SSP (Hagiwara, 2016).

A parallel development appears in LLM sparsification. CATS applies hard thresholding to the gated MLP activation

$\sigma^2$04

with a per-layer threshold $\sigma^2$05 calibrated to a desired sparsity level (Lee et al., 2024). The paper is explicit that base CATS is “scale-aware at the layer level” because the threshold depends on the empirical distribution of $\sigma^2$06 for that layer, but is static after calibration and does not adapt per token or per batch at inference time (Lee et al., 2024).

The reported performance trade-off is favorable at moderate sparsity. In zero-shot evaluation across eight tasks, Mistral-7B drops from $\sigma^2$07 to $\sigma^2$08 at CATS-$\sigma^2$09, and Llama2-7B drops from $\sigma^2$10 to $\sigma^2$11 (Lee et al., 2024). The paper also reports a custom GPU kernel that translates activation sparsity to approximately real-time speedups, including an overall wall-clock inference latency improvement of about $\sigma^2$12 and end-to-end throughput gains of about $\sigma^2$13 for Llama2-7B and $\sigma^2$14 for Mistral-7B at $\sigma^2$15 sparsity (Lee et al., 2024).

5. Variance-aware testing and scale-aware networks

In large-scale multiple testing, HART treats heteroscedasticity itself as the relevant scale variable. The conditional sampling model is

$\sigma^2$16

and the ranking statistic is the conditional local fdr

$\sigma^2$17

Because $\sigma^2$18 depends jointly on the observed statistic and its variance, HART “bypasses standardization” and exploits variance structure under the alternative that standardized $\sigma^2$19-values suppress (Fu et al., 2019). The data-driven threshold selects the largest $\sigma^2$20 such that the moving average of the sorted $\sigma^2$21 is at most $\sigma^2$22.

The simulation results in the paper illustrate the gain. In a toy example with $\sigma^2$23, $\sigma^2$24, and $\sigma^2$25, average power was $\sigma^2$26 for the p-value oracle, $\sigma^2$27 for the z-value oracle, and $\sigma^2$28 for the full-data oracle (Fu et al., 2019). In the myeloma microarray analysis at $\sigma^2$29, BH found $\sigma^2$30 discoveries, adaptive z-value found $\sigma^2$31, and HART found $\sigma^2$32 (Fu et al., 2019). The paper’s central claim is therefore not that standardization is always wrong, but that it can discard informative variance heterogeneity.

Network models provide another interpretation of scale-aware thresholding. In the classical hidden-variable model with exponential weights,

$\sigma^2$33

the expected degree satisfies

$\sigma^2$34

in the sparse regime, which implies

$\sigma^2$35

To keep the average degree $\sigma^2$36, the threshold must scale as

$\sigma^2$37

The paper generalizes this result and shows that broad classes of hidden-variable distributions and hard-threshold kernels still yield $\sigma^2$38 when an appropriate normalization condition is imposed (Balogh et al., 2019).

Softening the threshold with logistic kernels introduces tunable exponents. In the additive logistic case, the degree distribution obeys

$\sigma^2$39

for $\sigma^2$40, while the hard-threshold limit reverts to exponent $\sigma^2$41 (Balogh et al., 2019). Simulations with $\sigma^2$42 yielded $\sigma^2$43 at $\sigma^2$44, $\sigma^2$45 at $\sigma^2$46, $\sigma^2$47 at $\sigma^2$48, and $\sigma^2$49 at $\sigma^2$50 (Balogh et al., 2019).

Degree-thresholding renormalization studies thresholding at the graph level. DTR forms an induced subgraph $\sigma^2$51 by retaining nodes with degree $\sigma^2$52 from the original graph, and uses the relative size

$\sigma^2$53

as the scale variable (Chen et al., 2021). The finite-size scaling ansatz is

$\sigma^2$54

for families of networks with varying initial size, and

$\sigma^2$55

when snapshots are reused as original networks (Chen et al., 2021). For BA networks, a single exponent $\sigma^2$56 collapses both structural and dynamical observables; for single-size analyses, $\sigma^2$57 was reported for BA, CL snapshots, and the real networks studied (Chen et al., 2021). This suggests a renormalization interpretation of thresholding in which threshold choice is naturally indexed by retained relative size rather than by the raw degree cutoff alone.

6. Diagnostics, limitations, and recurring misconceptions

A major misconception is that any threshold chosen from data is automatically “adaptive” in the strong sense. The literature distinguishes several levels. ADT is dynamic at inference time because its action depends on current state variables derived from recent score statistics (Yang et al., 2023). CATS is contextually aware and scale-aware at the layer level, but the threshold is fixed after calibration and does not adapt per token at runtime (Lee et al., 2024). The KDE histogram method adapts via bandwidth search, but once the target number of classes is fixed, the thresholds are determined by minima of the selected KDE (Korneev et al., 2022).

A second misconception is that thresholding merely filters noise while leaving asymptotic structure intact. The birth–death analysis demonstrates the opposite: thresholding can impose a crossover scale and generate a spurious scaling regime with exponent $\sigma^2$58, obscuring the true $\sigma^2$59 tail unless sample sizes are very large (Font-Clos et al., 2014). The recommended remedy is explicit threshold-aware collapse across multiple thresholds rather than a single power-law fit.

A third recurring issue is granularity. Binary control in ADT may be too coarse when anomaly-score distributions overlap heavily (Yang et al., 2023). A common bandwidth in the KDE histogram method can produce spurious minima in long flat tails (Korneev et al., 2022). FITS$\sigma^2$60 requires non-overlapping groups and careful choice of $\sigma^2$61 and $\sigma^2$62 (Zhao et al., 8 Jan 2025). HART relies on accurate or consistently estimated $\sigma^2$63 and has $\sigma^2$64 kernel computations in its naive implementation (Fu et al., 2019). DTR may be unsuitable for homogeneous networks with narrow degree ranges (Chen et al., 2021).

Several papers also identify clear future directions. ADT explicitly mentions “continuous thresholding and multi-objective rewards” (Yang et al., 2023). The KDE thresholding paper points to variable-bandwidth KDEs, multidimensional histograms, and integration with spatial priors (Korneev et al., 2022). The CATS paper proposes scale-aware runtime variants in which $\sigma^2$65 depends on moving activation statistics or per-token RMS, while preserving the calibration target sparsity (Lee et al., 2024). These proposals suggest a broader trend: once thresholding is recognized as a scale-sensitive control mechanism, static cutoffs become special cases rather than the default formulation.