---
title: Shape-Aware Scaling Laws in Complex Systems
url: https://www.emergentmind.com/topics/shape-aware-scaling-laws
type: topic
---

# Shape-Aware Scaling Laws in Complex Systems

Shape-aware scaling laws formalize how performance, energy, or structural observables scale when the "shape"—not merely the size—of a system or model is varied. Here, "shape" can refer to architectural parameters in machine learning (width, depth, embedding dimension), geometric properties in physical systems, or combinatorial structure in networks. Incorporating variable shape reveals phenomena and scaling exponents inaccessible to classical, size-only scaling frameworks, producing sharper predictions and new design principles in domains such as deep learning, statistical mechanics, material science, and network theory.

## 1. Formalization of Shape-aware Scaling Laws

A shape-aware scaling law is an empirical or theoretical relationship connecting a performance metric (e.g., validation loss, friction force, energy minimum, width of a condensate, network diameter) not only to scalar measures like total system size $N$, dataset size $D$, or compute budget $C$, but also to specific "shape parameters" $s$—such as architectural width $w$ and depth $d$, geometric aspect ratios, or network motifs. The general form is
$$
O(N, D, s) \sim f(N, D)\, g(s)
$$
where $g(s)$ encodes "shape" sensitivity. For neural models, $s$ may be width $w$ and depth $d$; for spatial or network systems, $s$ may refer to geometric or topological descriptors.

Shape-aware scaling is established by:

- Varying shape parameters at fixed $N, D, C$ to empirically or analytically fit exponents in $g(s)$.
- Deriving the compute- or energy-optimal "shape" for a given overall system budget.
- Contrasting predictions for shape sensitivity in different problem domains (e.g., deep-wide preference in weather models vs. shape-insensitivity in language models).

Explicit shape-aware scaling laws enable model or system design choices that optimize performance or physical observables, given practical constraints.

## 2. Canonical Examples Across Domains

### Neural Weather Models

Yu et al. (2026) derive:
$$
L(N, D, w, d) \simeq A\, N^{-α} D^{-β} (w/w_0)^{-γ_w} (d/d_0)^{-γ_d}
$$
with exponents measured empirically for key weather models. Notably, at fixed parameter count $N$, loss falls steeply with width ($γ_w ≈ 0.45$) and weakly with depth ($γ_d ≈ 0.10$), inverting the "shape-insensitivity" observed in LLMs and yielding the recommendation that compute-optimal weather models should be "as wide as possible, with depth as small as architectural constraints allow" [2602.22962].

### Vision Transformers

Closed-form shape-aware scaling formulas are derived for each principal shape axis (width, depth, MLP size) in ViTs:
$$
f_k(x_k, t) \simeq α_k x_k^{-a_k} + (β_k x_k^{b_k} + ξ_k)\, t^{-c}
$$
Compute-optimal shapes allocate width, depth, and MLP dimension proportionally to $t^{s_k}$, with empirically fitted $s_{\text{depth}} ≈ 0.45$, $s_{\text{width}} ≈ 0.22$, $s_{\text{MLP}} ≈ 0.60$ [2305.13035].

### Condensation in Transport Processes

In stochastic transport with pair-factorized steady states (PFSS), condensate width $W$ exhibits shape- and regime-dependent scaling exponents:
$$
W \sim M_c^\alpha,\quad
\alpha =
\begin{cases}
0 & \text{(peak/single-site)}\\
\frac{\beta-\gamma}{\beta-\gamma+1} & \text{(rectangular plateau)}\\
\frac{\beta-\gamma}{2\beta-\gamma} & \text{(smooth/dome)}
\end{cases}
$$
where $\beta$ and $\gamma$ control surface and site-energy [1506.01641].

### Martensitic Microstructures

For elastic two-well problems, the minimal energy $E_{\min}$ of a martensitic nucleus in an austenitic matrix displays eight shape-dependent scaling regimes (including logarithmic and power-law dependencies in system geometry, modulus ratios, and surface energy), with phase diagrams classifying which microstructure (uniform, laminate, branching) is optimal as aspect ratios and parameters vary [2003.04095].

### Network Geometry and Empirical Systems

Structural network observables such as mean degree, clustering coefficient, mean geodesic distance, and assortativity exhibit domain-specific scaling exponents as functions of network size, directly linked to network "shape" features (degree heterogeneity, modularity, geometric embedding) [2603.20973, 1607.01435].

### Friction in Twisted Layered Interfaces

Static friction in finite polygonal or circular 2D flakes sliding on twisted hexagonal substrates admits distinct shape-dependent scaling:
- Circular: $F_s \sim R^{1/2}$ ($F_s \sim A^{1/4}$).
- Polygonal: $F_s$ is non-scaling (bounded, $\beta = 0$) for generic twist/edge orientations, but exhibits dual periodicity modulations at special alignments.
Thus, macroscale superlubricity can be engineered via shape and orientation design [2308.02818].

## 3. Methodologies and Fitted Exponents

The detection and quantification of shape-aware scaling laws rely on systematic measurement or simulation over controlled shape sweeps:

- For neural models, grid sweeps over $(w,d)$ or higher-dimensional tuples, "star sweep" perturbations, and log-log regression provide exponents $γ_w$, $γ_d$, $s_k$.
- In physical and mathematical models, a combination of analytic asymptotics (envelope/saddle-point approximations), rigorous upper/lower bounds, and Monte Carlo or molecular dynamics simulations yield scaling regimes and phase boundaries in space spanned by key geometric or energetic parameters.

Representative exponents for major applications:

| Domain                   | Shape Parameters          | Exponents (sample)            | Reference     |
|--------------------------|--------------------------|-------------------------------|--------------|
| Weather models           | width ($w$), depth ($d$) | $γ_w ≈ 0.45$, $γ_d ≈ 0.10$    | [2602.22962] |
| PFSS condensation        | $\beta$, $\gamma$        | see explicit formulas above   | [1506.01641] |
| Premixed flames          | $\alpha$, $\gamma$       | CH$_4$: $\alpha ≈ 0.036$, γ ≈ 0.98 | [2603.15592] |
| Neural force fields      | symmetry order, rep.     | $\alpha$ and $\beta$ up to 0.8 | [2510.09768] |

## 4. Contrasts and Domain-specific Sensitivities

A key insight from comparative studies is that shape sensitivity is deeply domain-dependent:

- **Language models**: Shape-insensitivity dominates (varying width vs. depth at fixed parameters moves loss by $<5\%$), with compute-optimal $N \propto C^{0.73}$ and $D \propto C^{0.27}$ [2602.22962].
- **Weather models**: Strong width preference; best practice is "shallow and wide," with a compute-optimal split prioritizing dataset size over parameter count, inverting the LLM prescription.
- **Frictional systems**: For generic polygons and incommensurate twist, friction scaling exponent $\beta=0$ (bounded friction), but specific edge-moiré alignments restore area scaling ($\beta = 1/2, 1, 1/4$).
- **Martensitic phase transformations**: Domain geometry selects between logarithmic and linear scaling in singular perturbation problems; only "highly compatible" polygons achieve the optimal (linear) scaling, whereas generic domains are penalized by $\epsilon |\log \epsilon|$ terms [2405.05927].

## 5. Mechanistic and Design Implications

Shape-aware scaling laws underpin actionable recommendations and sharpen the design of computational, physical, and experimental systems:

- **Neural net design**: Allocate parameter budget heavily to width for forecasting tasks; use minimal viable depth; scale data and model size in tandem for compute optimality [2602.22962, 2305.13035, 2510.09768].
- **Physical systems**: Engineer nucleation or pattern selection via geometric tailoring of domains to optimize energy scaling (e.g., nucleating martensitic variants in symmetry-adapted polygonal regions for sharp transitions).
- **Superlubricity via shape**: Cut 2D materials into generic polygons (edge-moiré misaligned) to saturate friction at finite value, or introduce curvature to achieve $\sim\, A^{1/4}$ scaling, thereby ensuring macroscale superlubricity [2308.02818].
- **Network modeling**: Validate generative models by matching empirically observed scaling exponents for network shape metrics; deviations signal missing mechanisms, such as degree heterogeneity or modularity [2603.20973].

## 6. Extensions, Adaptive Strategies, and Open Directions

Beyond static prescriptions, adaptive and schedule-based shape optimization further exploits scaling-law envelopes. For example, dynamic rescheduling of model shape (width, patch size, context length) during training allows models to traverse piecewise-optimal regimes on the error-compute curve, yielding substantial compute savings and tighter performance guarantees over static architectures [2311.03233].

The interaction of shape-aware scaling with inductive biases remains a fertile territory, as demonstrated by the substantial increase in scaling exponents ($\alpha, \beta$) when symmetry or equivariance constraints are imposed in neural potential models [2510.09768]. Similarly, mathematical theory continues to seek full classification of phase diagrams and shape-exponent transitions in high-dimensional and out-of-equilibrium systems.

In summary, shape-aware scaling laws unify disparate phenomena by illuminating the operational and physical significance of shape parameters, enabling precise optimization of performance or physical quantities across a wide spectrum of scientific and engineering domains.

Source: https://www.emergentmind.com/topics/shape-aware-scaling-laws