Joint Scaling Law in Complex Systems

• Joint scaling laws are defined as principled relationships that couple observables via power-law, lognormal, or universal forms, linking their scaling exponents.
• They are applied across disciplines, unifying insights in linguistics, genomics, neural scaling, and city-size hierarchies for improved predictive theory.
• Methodologies like data collapse and closed-form optimality validate these joint laws, revealing constraints that refine theory and model precision.

A joint scaling law expresses a principled relationship wherein two or more observable quantities in complex systems exhibit interdependent scaling behavior—typically via power-law, lognormal, or universal functional forms—whose exponents or parameters are not independent but are mathematically or mechanistically linked. Joint scaling laws unify phenomena that otherwise appear as separate or marginal scaling patterns, providing deeper insight into universality, constraint, and predictive theory across linguistics, genomics, neural scaling, statistical physics, and large-scale deep learning.

1. Conceptual Foundations and Formal Definition

A joint scaling law models the multivariate distribution or coupled evolution of observables, typically via an explicit relationship between their conditional (or marginal) distributions, leading to non-independent constraints on scaling exponents or collapse of data under universal forms. The hallmark is that knowledge of one scaling exponent (or law) constrains or determines the others through analytically tractable links. This framework generalizes classical scaling—e.g., $P(k) \propto k^{-q}$—to multidimensional or functionally coupled variables, such as (city) rank and class size, word frequency and vocabulary size, or model and data size.

A prototypical formulation is:

• Determine the joint probability (or moment) structure of variables $X$, $Y$ such that:

$$f_{X|Y}(x|y) = y^{-\alpha}\psi_X\left(\frac{x}{y^{\alpha}}\right), \qquad f_{Y|X}(y|x) = x^{-\beta}\psi_Y\left(\frac{y}{x^{\beta}}\right)$$

leading to bivariate functional equations and explicit linking of exponents or functional forms (Aoyama et al., 2010).

The joint scaling law is also formalized in multivariable risk or performance curves for complex models:

$L(N_1,N_2,\ldots) \sim \mathcal{F}(N_1,N_2,\ldots)$

where the scaling of loss $L$ as a function of several system parameters (e.g., model size, data size, architectural factors, data ratio) is not separable, but governed by a higher-level constraint (Zhao et al., 28 Sep 2025, Zhang et al., 10 Feb 2026, He et al., 2024).

2. Canonical Instances Across Disciplines

Joint scaling laws have been established across sciences; key exemplars with explicit mathematical relationships include:

• Quantitative Linguistics: The frequency distribution $P(n;L)$ of word types as a function of frequency $n$ and text length $L$ obeys a scaling ansatz

$P(n;L) = \frac{1}{L\,V_L}g\left(\frac{n}{L}\right)$

where $V_L$ (vocabulary size) is itself a function of $L$, and both Zipf's law and Heaps' law emerge as special cases linked through the behavior of the scaling function $g(x)$ (Font-Clos et al., 2013).

• Multivariate Production/ Econophysics: The bivariate distribution of firm sales $S$ and labor $L$ in Japanese firms satisfies two joint scaling laws, yielding a unique lognormal joint PDF determined by the exponents $\alpha$, $\beta$ of the conditional scaling, encoding a micro-macro equilibrium (Aoyama et al., 2010).
• Statistical Linguistics (Length–Frequency): The joint distribution of word length $\ell$ and frequency $n$ is found to be governed by a bivariate scaling form

$$P(n|\ell) = \ell^{\delta}\,G\left(n\,\ell^\delta\right)$$

and the coupled exponents $\alpha$, $\delta$ dictate the emergent Zipf exponent for the marginal $P(n)\sim n^{-\beta_z}$ with $\beta_z = \alpha + 1/\delta$ (Corral et al., 2019).

• City-Size Hierarchies: Zipf's law for city ranks connects to the hierarchical scaling law via two geometric/exponential intermediate relations, producing the equivalence

$N(m) = u\,P(m)^{-D},\quad D = \frac{1}{q}$

with $N(m)$ cities per class $m$, class-average size $P(m)$, and Zipf exponent $q$; thus, fractal dimension and distribution exponent are reciprocally tied (Chen, 2011).

• Genomic Evolution: The scaling of gene families and functional categories is coupled via the correlated duplication (recipe) model, predicting that the exponent $\beta_c$ of the family size distribution within functional category $c$ satisfies

$\beta_c = \frac{\beta}{\zeta_c}$

where $\zeta_c$ is the scaling exponent for the category's size with total genome size—superlinear growth enforces flatter distributions (Grilli et al., 2011).

• MoE and Multilingual Neural Scaling: Joint loss $L$ in LLMs or mixture-of-expert (MoE) architectures is expressed as a function of multiple coupled variables (e.g., model size, data size, number of experts, sampling ratios) with closed-form optimal configurations, universal exponents, and cross-factor dependencies (Zhao et al., 28 Sep 2025, He et al., 2024).

3. Mathematical Structure and Analytic Linkages

The core distinguishing feature is the presence of coupled or functional equations that relate the scaling of marginal and conditional distributions, often reducing the degrees of freedom relative to naive or independent power-law fits.

Examples:

• Linguistics (Zipf–Heaps Joint Law):

$$P(n;L) = \frac{1}{n\,[\,1+(n/n_a)^{\gamma-1}\,]} \ V_L = \frac{k}{a(\gamma-1)}\ln\left(1 + aL^{\gamma-1}\right)$$

with $n_a$ scaling linearly in $L$, so that the frequency and vocabulary growth exponents, and the transition between logarithmic and power-law growth, are analytically constrained by the exponents of $g(x)$ (Font-Clos et al., 2013).

• Bivariate Scaling and Productivity:

The double scaling law for $S$ and $L$ yields a bivariate lognormal with variance and correlation matrix determined by the scaling exponents, and the marginal distribution for productivity $r=S/L$ has scaling collapse determined by these indices (Aoyama et al., 2010).

• MoE Joint Law:

$$L(N,D,N_a,G,S) = \left(eG + \frac{f}{G} + mS^2+nS\right) \left(\frac{1}{N^\alpha} + \frac{k}{N_a^\alpha} + h\frac{N_a}{N}\right) + \frac{a}{N^\alpha} + \frac{b}{D^\beta} + \frac{c}{N_a^\alpha} + \epsilon$$

with optimal $G$, $S$, and $N_a/N$ determined analytically via minimization, showing nontrivial coupling across variables (Zhao et al., 28 Sep 2025).

• Multilingual Scaling:

$$L_i(N,D,p_i) = \left(E_i + \frac{A_i}{N^{\alpha_i}} + \frac{B_i}{D^{\beta_i}}\right)p_i^{-\gamma_i}$$

where the loss in language family $i$ depends only on the family’s data fraction $p_i$ and not on joint mixture, validated via controlled sweep experiments (He et al., 2024).

4. Methodologies, Empirical Confirmations, and Universality

Joint scaling laws are empirically validated by data collapse under rescaling transformations, quantitative fits of observable exponents, and universality under domain shifts and mechanism variations:

• Data collapse: For vocabulary growth, plotting $L\,V_L\,D_L(n)$ vs $n/L$ demonstrates single-master-curve behavior (Font-Clos et al., 2013).
• Closed-form optimality: For MoE systems, optimal configurations computed from joint laws robustly predict settings used in large-scale deployments (DeepSeek, Qwen, Kimi models) (Zhao et al., 28 Sep 2025).
• Universality: Transform invariance, mixture extensions, and transferability of exponents and functional forms are demonstrated across linearized NTK, finite-width, feature-learning, and neural scaling settings (Bi et al., 25 Sep 2025, He et al., 2024).
• Statistical physics analogy: The number of scaling indices is small, with macroscopic (aggregate) statistics emergent from the joint law, analogous to temperature and density defining equilibrium states (Aoyama et al., 2010).

5. Implications, Constraints, and Limitations

Joint scaling laws expose structural constraints in multivariate data: the measurement or modeling of one axis automatically specifies others, reducing ambiguity and increasing predictive power. This has implications for:

• Resource allocation: In neural scaling, joint exponents dictate optimal tradeoffs between compute, model size, and data (Ngo et al., 10 Oct 2025).
• Theory building: The existence of closed-form joint laws connects empirically observed scaling regimes to underlying stochastic or mechanistic models, as seen in genomics (coupled duplication/innovation), linguistics (unified statistical structure), and econophysics (micro–macro bridges) (Grilli et al., 2011, Font-Clos et al., 2013, Aoyama et al., 2010).
• Limitations: Functional forms and analytic links may break under regime changes, unmodeled coupling, nonpolynomial spectral tails, or when variables lose independence or the system departs from assumptions (e.g., very small data fractions, boundary artifacts, mis-tokenization, phase transitions in models) (Zhao et al., 28 Sep 2025, Bi et al., 25 Sep 2025, Chen, 2011).

6. Cross-Domain Extensions and Generalizations

Joint scaling law frameworks extend naturally to:

• Critical phenomena: Analogies are drawn with finite-size scaling and universality in statistical physics, where joint laws dictate phase behavior and finite-size corrections (Corral et al., 2019).
• Reinforcement of universality: Mixture-of-experts, multi-modal, and configuration-to-performance mapping in deep learning robustly fit into the joint-scaling paradigm, enabling predictive modeling of performance across multiple axes under large-scale heterogeneity (Zhang et al., 10 Feb 2026).
• Future directions: Open questions remain in understanding artifact regime transitions (largest outliers, foothill anomalies), interaction with quantization or radical architecture shifts, and the limiting behavior for highly non-polynomial or ultrahigh-dimensional data (Chen, 2011, Bi et al., 25 Sep 2025).

7. Representative Joint Scaling Laws Across Fields

Domain	Observables/Parameters	Joint Scaling Formulation	Key Constraint or Analytic Link
Linguistics	Word freq $n$, length $L$	$P(n;L) = g(n/L)/(L\,V_L)$	$V_L$ and $g(x)$ determine both Heaps' and Zipf's law
Econophysics	Sales $S$, labor $L$	$f_{S|L}(s|\ell)$ and $f_{L|S}(\ell|s)$	Exponents $\alpha, \beta$ uniquely determine joint and marginals
Multilingual LMs	$N$, $D$, $p_i$	$L_i(N,D,p_i)$ as explicit function in all variables	Loss per family depends only on $p_i$, exponents universal
City-size hierarchies	Rank $k$, class $m$	$P(k)\sim k^{-q}$ vs. $N(m)\sim P(m)^{-1/q}$	Exponents $q$, $1/q$ reciprocally tied via construction
Genomics	Family size $d$, category $c$	$f_c(d,n)\sim d^{-1-\beta_c}$, $n_c\sim n^{\zeta_c}$	$\beta_c = \beta/\zeta_c$ links evolutionary and functional exponents

These joint scaling laws demonstrate the universal underlying regularities that emerge from complex system interactions and foster trans-disciplinary advances in theory, modeling, and predictive analytics.