Scaling Laws in Complex Systems

• Scaling laws are mathematical relationships, typically in power-law form, that relate observable quantities to measures such as model size, dataset size, or compute, thereby revealing underlying system constraints.
• They are applied across fields—from machine learning and network science to astrophysics and biology—to predict performance changes and enable controlled extrapolation from accessible regimes.
• Recent research refines scaling laws using detailed exponent identities and redundancy measures, offering enhanced predictive accuracy and guiding optimal resource allocation.

Scaling laws are relations between an observable quantity and a measure of scale—system size, model size, dataset size, compute budget, mass, or characteristic length—most commonly in power-law form, $f(x) = c x^n$, with homogeneity $f(\lambda x) = \lambda^n f(x)$ under dilation (Muradyan, 2011). In contemporary usage, the term also covers logarithmic laws, joint multivariate laws with irreducible floors, and exponent identities among critical variables. Across the literature, scaling laws serve two closely related purposes: they expose the organizing constraints of a system, and they enable controlled extrapolation from accessible regimes to otherwise inaccessible ones (Rosenfeld, 2021).

1. Mathematical structure and canonical forms

The canonical scaling law is the power law $f(x) = c x^n$, where the exponent $n$ encodes how rapidly an observable changes under scale transformation (Muradyan, 2011). In machine learning, the corresponding empirical forms are typically written as a power-law decay of loss or error with respect to a resource variable, such as $e = k x^{-\alpha}$, or as a joint law such as

$$\epsilon(m, n) \approx a \cdot n^{-\alpha} + b \cdot m^{-\beta} + c_\infty,$$

where $m$ is model size, $n$ is dataset size, and $c_\infty$ is an error floor (Lin et al., 2024, Rosenfeld, 2021). In other settings, logarithmic forms are equally central: empirical networks exhibit $y = a + b \log_{10} n$ for mean geodesic distance and degree assortativity, while power laws remain appropriate for mean degree and clustering (Dutta et al., 21 Mar 2026).

A recurring structural feature is the presence of additive or asymptotic constants. In acoustic modeling, for example,

$f(\lambda x) = \lambda^n f(x)$0

and the combined law

$f(\lambda x) = \lambda^n f(x)$1

makes the irreducible loss explicit (Droppo et al., 2021). In critical phenomena, scaling laws may instead relate exponents to one another rather than an observable directly to scale, as in gravitational collapse, where exponent identities constrain the admissible critical behavior (Cai et al., 2015).

These forms are not interchangeable. A plausible implication is that the functional class—power law, logarithmic law, or exponent identity—tracks the dominant mechanism: self-similarity, small-world shortcutting, finite-resource saturation, or critical-point homogeneity.

2. Statistical learning theory and the origin of neural exponents

In deep learning, precise power-law relations between population loss and either dataset size or parameter count motivated a theoretical taxonomy distinguishing four regimes: variance-limited and resolution-limited scaling, each with respect to dataset size and model size (Bahri et al., 2021). In the variance-limited regime, the exponents are universal, with

$f(\lambda x) = \lambda^n f(x)$2

whereas in the resolution-limited regime the exponents depend on intrinsic manifold dimension and smoothness, yielding laws such as $f(\lambda x) = \lambda^n f(x)$3 and $f(\lambda x) = \lambda^n f(x)$4 (Bahri et al., 2021). In large-width or random-feature settings, kernel spectra furnish a dual description: if $f(\lambda x) = \lambda^n f(x)$5, then both $f(\lambda x) = \lambda^n f(x)$6 and $f(\lambda x) = \lambda^n f(x)$7 scale with exponent $f(\lambda x) = \lambda^n f(x)$8 (Bahri et al., 2021).

A more recent formalization identifies scaling laws as redundancy laws. Under kernel regression with covariance eigenvalues $f(\lambda x) = \lambda^n f(x)$9, the expected excess risk obeys

$f(x) = c x^n$0

where $f(x) = c x^n$1 is a source-condition smoothness parameter and $f(x) = c x^n$2 is the redundancy index (Bi et al., 25 Sep 2025). The exponent is therefore non-universal: flatter spectra imply higher redundancy and slower returns to scale, while steeper spectra increase $f(x) = c x^n$3 (Bi et al., 25 Sep 2025).

Linear regression theory provides an explicit reconciliation between empirical neural scaling laws and the classical bias–variance decomposition. In an infinite-dimensional linear model trained by one-pass SGD, the reducible test error is

$f(x) = c x^n$4

under a power-law covariance spectrum $f(x) = c x^n$5 with $f(x) = c x^n$6 (Lin et al., 2024). The variance term, although it increases with model size in conventional analyses, is dominated by the other errors because of the implicit regularization of SGD, and therefore disappears from the dominant bound (Lin et al., 2024). This suggests that monotone neural improvement with model size is not a contradiction of classical theory so much as a consequence of a particular optimization regime.

The same literature extends beyond dense training. Iterative magnitude pruning admits a joint law in width, depth, density, and data, together with an error-preserving invariant that permits width, depth, and pruning density to be interchanged at fixed error (Rosenfeld, 2021). That result locates pruning within the same scaling-law framework as dense training rather than outside it.

3. Large-scale AI systems and compute-optimal scaling

The most direct large-model comparison in the supplied literature concerns linear-complexity LLMs. Three efficient architectures—TNL, HGRN2, and cosFormer2—were compared with LLaMA after training six model sizes from $f(x) = c x^n$7M to $f(x) = c x^n$8B parameters on $f(x) = c x^n$9B tokens, with $n$0 intermediate checkpoints across validation loss, commonsense reasoning, long-context retrieval, and generation (Shen et al., 2024). All families obeyed compute-based power laws of the form

$n$1

with closely matched loss exponents, approximately $n$2 to $n$3, and linear models supported slightly larger optimal model and data sizes at fixed compute (Shen et al., 2024). The same study found that existing linear-complexity LLMs exhibit similar scaling capabilities as conventional transformer-based models while also demonstrating superior linguistic proficiency and knowledge retention, yet transformers remain significantly better on Needle In A Haystack retrieval, especially in very long contexts (Shen et al., 2024).

Code understanding exhibits the same phenomenology. For masked-language-model pretraining on code, the test error follows

$n$4

with $n$5 taken as model size, training data, or compute (Lin et al., 2024). The study scaled transformer encoders from $n$6M to $n$7B parameters and trained CoLSBERT with $n$8B parameters on approximately $n$9GB and $e = k x^{-\alpha}$0B training tokens, using $e = k x^{-\alpha}$1 A100-40G GPUs, and reported higher Code Search MRR and Clone Detection MAP than prior $e = k x^{-\alpha}$2M-parameter baselines (Lin et al., 2024). Acoustic models trained with auto-predictive coding show analogous laws, and on $e = k x^{-\alpha}$3-layer transformers the fitted constants were $e = k x^{-\alpha}$4, $e = k x^{-\alpha}$5, $e = k x^{-\alpha}$6, $e = k x^{-\alpha}$7, and $e = k x^{-\alpha}$8 (Droppo et al., 2021).

Scaling laws also organize reinforcement learning. In AlphaZero, playing strength scales as

$e = k x^{-\alpha}$9

with $$\epsilon(m, n) \approx a \cdot n^{-\alpha} + b \cdot m^{-\beta} + c_\infty,$$0 for Connect Four and $$\epsilon(m, n) \approx a \cdot n^{-\alpha} + b \cdot m^{-\beta} + c_\infty,$$1 for Pentago, and $$\epsilon(m, n) \approx a \cdot n^{-\alpha} + b \cdot m^{-\beta} + c_\infty,$$2 for both games (Neumann et al., 2022). Combining these yields an optimal-size law

$$\epsilon(m, n) \approx a \cdot n^{-\alpha} + b \cdot m^{-\beta} + c_\infty,$$3

with empirical exponents $$\epsilon(m, n) \approx a \cdot n^{-\alpha} + b \cdot m^{-\beta} + c_\infty,$$4 and $$\epsilon(m, n) \approx a \cdot n^{-\alpha} + b \cdot m^{-\beta} + c_\infty,$$5 (Neumann et al., 2022). The same work reports that larger AlphaZero models are more sample efficient (Neumann et al., 2022). For transformer development, small-scale experiments can be predictive, but only conditionally: some fine-tuning tasks exhibit clear power laws, and extrapolation was found useful when $$\epsilon(m, n) \approx a \cdot n^{-\alpha} + b \cdot m^{-\beta} + c_\infty,$$6, yet careful hyperparameter tuning and multiple runs for uncertainty estimation were necessary and partially offset the computational savings (Ivgi et al., 2022).

The idea generalizes to problem size. In Hex, AlphaZero frontiers across board sizes showed that performance achievable with a fixed amount of compute degrades predictably as the game gets larger and harder, and train-time and test-time compute can be traded off almost linearly in log-space while maintaining performance (Jones, 2021). A plausible implication is that compute-optimal scaling is not restricted to fixed-task asymptotics.

4. Networks, geometry, and criticality

In spatial network formation, geometric constraints change the scaling class itself. When contacts are added under excluded-volume and planarity constraints, graph diameter $$\epsilon(m, n) \approx a \cdot n^{-\alpha} + b \cdot m^{-\beta} + c_\infty,$$7 and radius of gyration $$\epsilon(m, n) \approx a \cdot n^{-\alpha} + b \cdot m^{-\beta} + c_\infty,$$8 scale algebraically,

$$\epsilon(m, n) \approx a \cdot n^{-\alpha} + b \cdot m^{-\beta} + c_\infty,$$9

rather than logarithmically as in unconstrained random networks (Molkenthin et al., 2016). The exponent lies between the space-filling and self-avoiding-random-walk limits; experimentally measured protein tertiary structures in $m$0D give $m$1, while the $m$2D model gives $m$3 (Molkenthin et al., 2016). The change from $m$4 to $m$5 is not a minor quantitative correction but a change in universality class induced by geometry (Molkenthin et al., 2016).

Empirical network science displays a mixed picture of universal functional forms and domain-specific rates. Across $m$6 real-world networks from social, biological, informational, and technological domains, mean degree scales as $m$7, mean geodesic distance as $m$8, clustering as $m$9, and degree assortativity as $n$0 (Dutta et al., 21 Mar 2026). The domains share these functional forms, but the fitted exponents differ: social networks densify fastest, biological networks show the steepest logarithmic growth in path length, and technological networks show the fastest decay in clustering (Dutta et al., 21 Mar 2026). Configuration-model random graphs reproduce the scaling behavior remarkably well, while degree-corrected stochastic block models are slightly better (Dutta et al., 21 Mar 2026).

Critical gravitational collapse supplies yet another use of scaling laws: relations among critical exponents. For spherically symmetric collapse, the exponents satisfy

$n$1

independently of the details of the gravity theory under consideration (Cai et al., 2015). These identities are explicitly compared to scaling relations in thermodynamic and geometrical phase transitions (Cai et al., 2015). The analogy does not imply identical microscopic dynamics; it does imply a shared homogeneity structure near criticality.

5. Biological and biomedical allometry

A recent biological program derives scaling laws from invariance of a macroscale system of coupled partial differential equations for fluid dynamics, nonlinear elasticity, and rigid body mechanics (Liu et al., 17 Feb 2025). Under the transformation from one organism to another, length scales as $n$2, time as $n$3, density as $n$4, elastic modulus as $n$5, viscosity as $n$6, and gravity as $n$7, while the Reynolds, Froude, Strouhal, and Elastic Mach numbers are preserved (Liu et al., 17 Feb 2025). For large, geometrically similar animals in the inertial regime,

$n$8

and

$n$9

(Liu et al., 17 Feb 2025). In microorganisms at low Reynolds number, the relevant laws reverse direction, with $c_\infty$0 and $c_\infty$1 (Liu et al., 17 Feb 2025).

Cancer scaling departs sharply from classical sublinear metabolic laws. Across several malignant histologies, total lesion activity and metabolic tumor volume obey

$c_\infty$2

with superlinear exponents in the approximate range $c_\infty$3–$c_\infty$4, typically clustering around $c_\infty$5 (Pérez-García et al., 2021). The corresponding phenomenological growth model,

$c_\infty$6

implies explosive, super-exponential volumetric growth for $c_\infty$7 (Pérez-García et al., 2021). The mechanistic account invokes evolutionary dynamics among subclones, and the associated nonlocal Fisher–Kolmogorov analysis yields $c_\infty$8, so only in three dimensions does the theory produce the superlinear value $c_\infty$9 (Pérez-García et al., 2021). The paper further defines a Distance to Scaling Law metric,

$y = a + b \log_{10} n$0

and reports prognostic value for deviations above or below the fitted allometric curve (Pérez-García et al., 2021).

These biological examples underscore that a scaling exponent may encode dominant transport physics, elasticity, geometry, or evolutionary competition rather than a generic notion of “size effect.”

6. Physical sciences: from hadrons to coronal loops

In particle physics, scaling laws often express constituent structure. The dimensional quark counting rules give

$y = a + b \log_{10} n$1

for electromagnetic form factors of a composite object with $y = a + b \log_{10} n$2 constituents, and for exclusive two-body reactions $y = a + b \log_{10} n$3,

$y = a + b \log_{10} n$4

where $y = a + b \log_{10} n$5 (Muradyan, 2011). The same paper also discusses generalized spin–mass relations, including

$y = a + b \log_{10} n$6

for string-, disk-, and ball-like morphologies, and the Kerr bound

$y = a + b \log_{10} n$7

for rotating black holes (Muradyan, 2011).

Planetary-dynamo theory uses power laws to connect hidden core dynamics to observable dipole moments and rotation rates. Under the assumption that the nonlinear inertial term is negligible because of the small planetary Rossby number, the proposed laws include

$y = a + b \log_{10} n$8

together with $y = a + b \log_{10} n$9 and a convection-column aspect ratio scaling as $f(\lambda x) = \lambda^n f(x)$00 (Davidson, 2013). The novelty relative to earlier dynamo scalings is the explicit exclusion of inertial balance from the derivation (Davidson, 2013).

Solar-loop thermodynamics furnish a further example of a scaling law surviving a change in physical regime. The classic static coronal-loop laws are

$f(\lambda x) = \lambda^n f(x)$01

and for collision-dominated thermal conduction the dynamic extension preserves the same functional dependencies on pressure and length, while replacing the constants of proportionality by Mach-number-dependent factors (Bradshaw et al., 2020). In turbulence-limited conduction, by contrast, the temperature dependence of the conductivity changes enough that the scaling laws depart from their static forms and the return enthalpy flux acquires a limiting Mach number set by the turbulent mean free path (Bradshaw et al., 2020).

7. Predictive use, universality, and caveats

Scaling laws are attractive because they convert expensive search into measured extrapolation. In particle-physics amplitude surrogates, the test loss is modeled as

$f(\lambda x) = \lambda^n f(x)$02

with $f(\lambda x) = \lambda^n f(x)$03 taken as network size, training data, or compute (Bahl et al., 19 Jan 2026). The exponents are linked to the intrinsic dimension $f(\lambda x) = \lambda^n f(x)$04 of the phase-space manifold, and the paper reports the estimate $f(\lambda x) = \lambda^n f(x)$05 as a practically useful rule for reaching predefined precision targets (Bahl et al., 19 Jan 2026). In jet classification, six physically motivated classifiers obey

$f(\lambda x) = \lambda^n f(x)$06

but with distinct exponents—$f(\lambda x) = \lambda^n f(x)$07 for kNN EMD, $f(\lambda x) = \lambda^n f(x)$08 for DNN on LOT, down to $f(\lambda x) = \lambda^n f(x)$09 for kNN EFP in the large-$f(\lambda x) = \lambda^n f(x)$10 regime—so the best classifier can change as the dataset is scaled (Batson et al., 2023). That finding makes fixed-dataset comparisons potentially misleading (Batson et al., 2023).

The same predictive logic appears in small-scale transformer experiments and board games. For language understanding tasks that exhibit clear scaling laws, larger-model performance can be predicted from much smaller models, but only after scale-specific hyperparameter tuning and hierarchical-bootstrap uncertainty estimation (Ivgi et al., 2022). In Hex, laws fitted on small boards predict larger and harder boards, and the residual prediction error drops exponentially as more small board sizes are included in the fit (Jones, 2021). These are operational successes, not merely descriptive ones.

At the same time, the literature repeatedly rejects a single universal exponent. Neural exponents vary across variance-limited, resolution-limited, and redundancy-controlled regimes (Bahri et al., 2021, Bi et al., 25 Sep 2025). Network statistics share functional forms across domains while differing in rate parameters (Dutta et al., 21 Mar 2026). Linear-complexity LLMs match transformers on many linguistic measures yet remain weaker at sparse long-context retrieval (Shen et al., 2024). A careful summary is therefore that scaling laws are widespread, but the exponent is usually a compressed statement of domain-specific structure—data redundancy, manifold dimension, transport regime, geometric constraint, modularity, or evolutionary dynamics—rather than a field-independent constant.