Deviation-Aware Scaling (DAS)

• Deviation-Aware Scaling (DAS) is a framework that adaptively modulates scaling in systems with heterogeneous fluctuations using explicit deviation measurements.
• It translates large deviation theory and norm imbalance analyses into practical algorithms that improve robustness and computational efficiency.
• DAS applies across domains such as deep learning, urban scaling, and high-energy physics, enabling adaptive resource allocation and improved generalization.

Deviation-Aware Scaling (DAS) refers to a suite of methodologies, theoretical frameworks, and practical algorithms designed to adaptively modulate scaling behavior in systems where fluctuations, heterogeneity, or outlier effects play a critical role. In diverse domains—including stochastic interacting particle systems, high-energy physics, urban scaling, deep learning model optimization, and out-of-distribution (OOD) detection—DAS leverages explicit measurements of deviation, dispersion, or norm imbalance to inform and optimize system parameters. This approach leads to enhanced robustness, improved generalization, and principled resource allocation by accounting for the underlying probabilities or dynamical statistics of rare events or structural imbalance.

1. Mathematical Principles and Rate Function Foundations

Deviation-Aware Scaling is fundamentally grounded in large deviation theory, stochastic control, and statistical mechanics. In interacting particle systems subject to both individual and common sources of noise, the probability of atypical deviations from the mean-field limit is governed by a rate function whose form is determined by the scaling regime of the common noise intensity parameter $K(n)$ (Budhiraja et al., 2019). Two central asymptotic regimes are distinguished:

• Balanced regime ($K(n) \sim n^{-1/2}$):

$$I_1(v) = \inf_{y \in L^2,\; \Theta \in \mathcal{E}_1[v]} \left\{ \frac{1}{2} \int_0^T \|y(t)\|^2\,dt + \frac{1}{2} \int_0^T \|u^+(t)\|^2\,dt \right\}$$

where $y(t)$ models control against common noise and $u^+(t)$ quantifies deviation cost from individual noise sources.

• Weak or dominant common noise regimes:

If $V_n K(n) \to 0$, the common noise is negligible and the deviation cost is inherited solely from individual control contributions. Conversely, for $V_n K(n) \to \infty$, the common noise dominates, leading to slower Laplace asymptotics and modified rate functions.

Within urban scaling and allometric growth, the theoretical relation for scaling exponents $a_{ji}$ between logarithmic measures $Q_i(t)$ and $Q_j(t)$ is (Chen, 2020): $a_{ji} = \frac{\sigma_i}{\sigma_j}$ where $\sigma_i$ and $\sigma_j$ denote the standard deviations of $\ln Q_i(t)$ and $\ln Q_j(t)$, respectively. Empirically, this adjusts to: $a_{ji}^* = R \cdot \frac{s_i}{s_j}$ with $R$ as the Pearson correlation coefficient, capturing deviation-aware adjustments in scaling laws.

2. Mechanisms and Algorithms in Practice

DAS translates foundational rate function calculations and norm deviation analyses into algorithms capable of real-time system adaptation. In optimization for tensorized models and scale-invariant architectures, DAS can replace the computationally expensive adversarial perturbation of Sharpness-Aware Minimization (SAM) (Cao et al., 14 Aug 2025) with an explicit scaling update step:

• Norm Deviation Metric:

$$Q = \sum_{k} \left(\|\mathcal{G}_k\|_F^2 - \frac{1}{K}\sum_{i} \|\mathcal{G}_i\|_F^2 \right)^2$$

where $\mathcal{G}_k$ represents the $k$-th tensor core.

• Scaling Update:

$$\lambda_k^{(t)} = \frac{\eta\alpha u^{(t)}}{\|\mathcal{G}_k^{(t)}\|_F^2}\left(\|g_k^{(t)}\|_F^2 - \overline{g^2}\right)$$

where $\eta$ is the learning rate, $\alpha$ adjusts regularization strength, and $\overline{g^2}$ is the mean squared gradient norm.

This explicit update reduces computational overhead relative to SAM, while retaining the regularizing effects on core norm balancing.

In post-hoc OOD detection for deep learning models, DAS leverages deviation in activation shifts under small perturbations to adjust sample-specific scaling thresholds (Regmi, 11 Mar 2025):

• Activation Shift-Based OOD Metric:

$$Q = \sum_{j \in \text{argsort}(\mathbf{a})[:k_1]} |a_j^\varepsilon - a_j|$$

where $\mathbf{a}$ are activations from input $x$, and $a_j^\varepsilon$ are from perturbed input $x^\varepsilon$.

• Adaptive Percentile Scaling:

$$p = p_{\min} + (1 - F_{Q'}(Q')) \cdot (p_{\max} - p_{\min})$$

where $Q'$ is a corrected OOD likelihood metric, and $F_{Q'}(\cdot)$ is an empirical CDF normalization.

This mechanism demonstrates significant improvements in FPR@95, maintaining in-distribution accuracy while enhancing reliability in OOD scenarios.

3. Regimes, Constraints, and System-Level Implications

Deviation-Aware Scaling must be tuned according to the dominant noise sources, system heterogeneity, and specific application constraints. For stochastic particle systems, the scaling regime (value of $K(n)$ relative to $n^{-1/2}$) uniquely determines whether individual or global noise contributions drive large deviation probabilities (Budhiraja et al., 2019). Control-theoretic representations map directly onto practical resource allocation rules, e.g. ensuring the probability of system overload remains below a target: $$P(\mu_n \in \text{undesired region}) \sim \exp(-n I(\text{undesired})) \leq \epsilon$$

In tensorized model training, norm deviation dynamics inform the choice of scaling strength and regularization, with empirical covariance between norm and gradient magnitudes governing the direction and rate of imbalance correction.

4. Generalization Across Disciplines

DAS frameworks generalize beyond their original context, yielding unified principles applicable to:

• High-energy physics, where the “double-asymptotic scaling” approach (DAS) enables analytical extraction of transverse-momentum dependent parton densities (TMDs) in Quantum Chromodynamics (Kotikov et al., 2019).
• Urban systems, with scaling exponents, fractal dimensions, and Zipf law parameters uniformly expressible as ratios of the standard deviations of relevant logarithmic measures (Chen, 2020).

This universality underlines DAS as a fundamental scaling paradigm, with dispersion-awareness at its core.

5. Experimental Validation and Performance Metrics

Empirical results consistently validate DAS methodologies:

• In OOD detection on ImageNet-1k across eight architectures, adaptive deviation-aware scaling yields average improvements over OptFS of 14.94 (near-OOD) and 21.67 (far-OOD) in FPR@95 (Regmi, 11 Mar 2025).
• Tensor completion experiments show DAS attaining $R^2$ scores comparable to SAM, with both finding flatter minima than traditional algorithms, and DAS providing competitive performance with substantial reductions in runtime (Cao et al., 14 Aug 2025).

Comparison of approaches is summarized:

Application	DAS Benefit	Computational Cost
OOD Detection (AdaSCALE)	Improved FPR@95, robust ID	Minimal ID data needed
Tensorized Model Optimization	Comparable SAM performance	No adversarial gradient
Urban Scaling Analysis	Exact scaling exponents	Standard deviation ratio

6. Relevance, Limitations, and Potential Extensions

Deviation-Aware Scaling mechanisms offer distinct advantages in scalability, interpretability, and computational cost, especially valuable in large-system and real-time settings. Nevertheless, precise DAS formulations depend on the accurate estimation of deviation metrics, normalization constants, and, in control-theoretic frameworks, solution of value functions which may require nontrivial computational resources in high-dimensional systems.

A plausible implication is that future research will seek to distill further norm-control strategies, extend deviation-aware regularizers to additional structured models, and optimize DAS for even more complex regimes in learning and control.

7. Interpretative Summary

In conclusion, Deviation-Aware Scaling encapsulates a principled, theoretically rigorous, and empirically validated framework for adaptive resource allocation, robust training, and reliable prediction in systems where dispersion, deviation, and rare events shape behavior. By linking scaling laws, stochastic control, norm dynamics, and deviation metrics across diverse domains, DAS enables the design of robust algorithms and system dynamics attuned to the underlying statistics of heterogeneity and fluctuation. This suggests an enduring role for DAS in both foundational research and engineering applications where adaptive scaling is essential to system performance and generalization.