HeteroScale: Asymmetric Multiscale Design

• HeteroScale is a design principle that explicitly defines and leverages asymmetry across different scales to improve structure and resource allocation in complex systems.
• It decouples roles among scales—for example, large-scale perception versus small-scale aggregation—enabling enhanced efficiency in vision models and network analysis.
• The concept spans diverse applications from hypergraphs and weighted networks to LLM inference and metamaterials, while raising open challenges in model tuning and dynamic scaling.

Searching arXiv for recent and directly relevant papers on “HeteroScale” and adjacent usages of the term. HeteroScale denotes a family of technical ideas in which heterogeneity across scales is made explicit and operational rather than treated as noise or implementation detail. In recent literature, the label is used in several distinct but related senses: simultaneous scale-free structure in hypergraphs, entropy-based selection of an optimal weight scale in weighted networks, decoupled large-scale perception and small-scale aggregation in lightweight vision models, coordinated autoscaling for heterogeneous Prefill/Decode-disaggregated LLM inference, and scale-rich mechanical architectures whose geometry and connectivity vary over orders of magnitude (Li et al., 2023, Garlaschelli et al., 2013, Wang et al., 29 Mar 2025, Li et al., 27 Aug 2025, Both et al., 22 Nov 2025). Across these usages, the common theme is not generic multiscale analysis, but controlled asymmetry between scales, roles, or resources.

1. Scope and principal meanings

Across the cited literature, “HeteroScale” is not a single standardized formalism. It functions instead as a recurrent design and analysis motif, instantiated differently in network science, machine learning, distributed systems, and metamaterials.

Domain	Technical meaning	Source
Hypergraphs	Bi-heterogeneity in node hyperdegree and hyperedge size	(Li et al., 2023)
Weighted networks	Optimal scale of link intensities via maximum entropy	(Garlaschelli et al., 2013)
Vision models	Large-scale perception with small-scale aggregation	(Wang et al., 29 Mar 2025)
LLM serving	Coordinated autoscaling for heterogeneous P/D-disaggregated inference	(Li et al., 27 Aug 2025)
Metamaterials	Scale-rich networks with broad variability in length, thickness, and degree	(Both et al., 22 Nov 2025)

Additional usages extend the same motif. In percolation, cluster heterogeneity $H$ is defined as the number of distinct cluster sizes and obeys its own scaling law and finite-size exponent (Noh et al., 2011). In institutional science, heterogeneous densification refers to institution-specific superlinear collaboration exponents $c_i(n) \sim n^{\alpha_i}$ together with Heaps’ and Zipf-like laws for the institutional landscape (Burghardt et al., 2020). In heterophilous graph learning and heterogeneous network embedding, multiscale and typed structural operators are introduced precisely because local one-hop homophily assumptions fail (Li et al., 2023, Guo et al., 2022).

2. Network-science and statistical formulations

A canonical structural use of HeteroScale appears in the Double Heterogeneous Hypergraph model, where preferential attachment acts at two levels: nodes and hyperedges. The model generates a power-law node hyperdegree distribution $P(d_H) \sim d_H^{-\gamma_d}$ and a power-law hyperedge-size distribution $P(S) \sim S^{-\gamma_e}$, with exponents

$$\gamma_d = 2 + \frac{\lambda}{p}, \qquad \gamma_e = 1 + \frac{\lambda + p}{\lambda(1-p)}, \qquad \lambda = \frac{m_{new}}{m_{old}}.$$

The two exponents are coupled, and tuning $p$ drives them in opposite directions: increasing node heterogeneity decreases hyperedge heterogeneity, and vice versa. The mechanism is dual preferential attachment: high-hyperdegree nodes are more likely to join newly formed hyperedges, while large hyperedges are more likely to absorb new nodes (Li et al., 2023).

A different statistical meaning appears in weighted networks. There the problem is not simultaneous power laws, but arbitrariness in the choice of weight units and in the generalization of binary observables. The proposed resolution maps a weighted network $W=\{w_{ij}\}$ to an ensemble of binary graphs through

$p(w_{ij}, z) = \frac{z w_{ij}}{1 + z w_{ij}},$

with $z$ acting as a scale parameter. The Shannon entropy

$$S(z) = -K\sum_{i,j}\big[p_{ij}\ln p_{ij} + (1-p_{ij})\ln(1-p_{ij})\big]$$

has a unique maximum at $c_i(n) \sim n^{\alpha_i}$0, yielding the optimal weight unit $c_i(n) \sim n^{\alpha_i}$1 and dimensionless weights $c_i(n) \sim n^{\alpha_i}$2. This makes the resulting weighted observables invariant to arbitrary changes of measurement units (Garlaschelli et al., 2013).

Percolation furnishes a third formulation. Cluster heterogeneity $c_i(n) \sim n^{\alpha_i}$3, defined as the number of distinct cluster sizes in a configuration, diverges near the critical point as

$c_i(n) \sim n^{\alpha_i}$4

and its finite-size scaling is controlled not by $c_i(n) \sim n^{\alpha_i}$5 but by

$c_i(n) \sim n^{\alpha_i}$6

The resulting scaling form

$c_i(n) \sim n^{\alpha_i}$7

identifies a singular path to criticality through the maximum of $c_i(n) \sim n^{\alpha_i}$8 (Noh et al., 2011).

At the level of social organization, heterogeneous scaling takes the form of institution-specific superlinear densification. The number of collaborations at institution $c_i(n) \sim n^{\alpha_i}$9 obeys $P(d_H) \sim d_H^{-\gamma_d}$0, with exponents $P(d_H) \sim d_H^{-\gamma_d}$1 that vary significantly across institutions. At the system level, the number of institutions follows Heaps’ law $P(d_H) \sim d_H^{-\gamma_d}$2, while institution sizes follow a Zipf-like law $P(d_H) \sim d_H^{-\gamma_d}$3. In the proposed model, these arise from three coupled mechanisms: researchers collaborate with friends-of-friends, new institutions trigger more potential institutions, and researchers are preferentially hired by large institutions (Burghardt et al., 2020).

3. Representation learning and vision architectures

In efficient vision, HeteroScale appears as an explicit decoupling of perception and aggregation scales. LSNet formulates standard token mixing as

$P(d_H) \sim d_H^{-\gamma_d}$4

then replaces it with the hetero-scale form

$P(d_H) \sim d_H^{-\gamma_d}$5

Its LS convolution implements Large-Kernel Perception and Small-Kernel Aggregation: a large-kernel depthwise path produces context-adaptive weights, and a small dynamic grouped convolution performs local aggregation. The resulting complexity is

$P(d_H) \sim d_H^{-\gamma_d}$6

which is linear in spatial size $P(d_H) \sim d_H^{-\gamma_d}$7. In the reported ImageNet-1K results, LSNet-B achieves $P(d_H) \sim d_H^{-\gamma_d}$8 at $P(d_H) \sim d_H^{-\gamma_d}$9G FLOPs, and the ablations show that removing either Large-Kernel Perception or Small-Kernel Aggregation degrades performance (Wang et al., 29 Mar 2025).

An analogous multiscale logic appears in heterophilous graph learning. PEGFAN constructs Haar-type graph framelets over a hierarchical clustering and proves conditions under which the resulting set is a tight frame. It then uses multi-hop channels $P(S) \sim S^{-\gamma_e}$0 together with framelet projections $P(S) \sim S^{-\gamma_e}$1 or $P(S) \sim S^{-\gamma_e}$2, preserving permutation equivariance while extracting coarse and fine graph-scale components. The empirical result is strongest on larger, denser heterophilous benchmarks such as Chameleon and Squirrel, where the framelet-augmented model outperforms strong multi-hop baselines (Li et al., 2023).

In heterogeneous network embedding, HAW and CHAW provide a scalable representation of typed local structure. A heterogeneous anonymous walk augments anonymous-walk positions with node types; the number of possible HAWs of length $P(S) \sim S^{-\gamma_e}$3 is $P(S) \sim S^{-\gamma_e}$4, where $P(S) \sim S^{-\gamma_e}$5 is the Bell number. The authors show that the heterogeneous radius-$P(S) \sim S^{-\gamma_e}$6 neighborhood $P(S) \sim S^{-\gamma_e}$7 can be reconstructed from the distribution of HAWs of length $P(S) \sim S^{-\gamma_e}$8, then introduce the coarser CHAW representation and the HAWE/CHAWE embedding framework to make learning feasible at scale (Guo et al., 2022).

4. Systems, autoscaling, and data infrastructure

The most literal use of the term is the coordinated autoscaling framework "HeteroScale" for P/D-disaggregated LLM inference. The setting is ByteDance’s Seed Serving Platform, where Prefill and Decode run on different worker pools, GPU hardware is heterogeneous, and KV-cache transfer is sensitive to RDMA topology. HeteroScale couples a topology-aware scheduler with a metric-driven autoscaling policy centered on decode Tokens Per Second. The primary proportional-control rule is

$P(S) \sim S^{-\gamma_e}$9

and the resulting target is then split into prefill and decode counts using a fixed optimal P/D ratio. The scheduler operates over Deployment Groups and RDMA Subgroups, preserving network affinity while allocating heterogeneous GPU resources. In production on tens of thousands of GPUs, the framework increased average GPU utilization by $$\gamma_d = 2 + \frac{\lambda}{p}, \qquad \gamma_e = 1 + \frac{\lambda + p}{\lambda(1-p)}, \qquad \lambda = \frac{m_{new}}{m_{old}}.$$0 percentage points and saved hundreds of thousands of GPU-hours daily, while maintaining TTFT and TBT service-level objectives (Li et al., 27 Aug 2025).

A closely related systems realization is HSPMD in Hetu v2. HSPMD extends standard SPMD with Device Group Union, Distributed States Union, HDim, and HSize so that asymmetric sharding, hierarchical communication, and dynamic graph switching can be expressed in a single-device declarative model. Progressive graph specialization then removes non-local operators, substitutes explicit communication operators with collectives or batched send-receive, and produces device-specific executable graphs. This supports both spatial heterogeneity, through device-specific execution logic, and temporal heterogeneity, through multiple annotation sets and fused BSR-based state migration (Li et al., 29 Apr 2025).

At the level of queueing and dispatch, scalable heterogeneity-aware control is formalized by JIQ-$$\gamma_d = 2 + \frac{\lambda}{p}, \qquad \gamma_e = 1 + \frac{\lambda + p}{\lambda(1-p)}, \qquad \lambda = \frac{m_{new}}{m_{old}}.$$1 and JSQ-$$\gamma_d = 2 + \frac{\lambda}{p}, \qquad \gamma_e = 1 + \frac{\lambda + p}{\lambda(1-p)}, \qquad \lambda = \frac{m_{new}}{m_{old}}.$$2. Each arrival probes $$\gamma_d = 2 + \frac{\lambda}{p}, \qquad \gamma_e = 1 + \frac{\lambda + p}{\lambda(1-p)}, \qquad \lambda = \frac{m_{new}}{m_{old}}.$$3 fast and $$\gamma_d = 2 + \frac{\lambda}{p}, \qquad \gamma_e = 1 + \frac{\lambda + p}{\lambda(1-p)}, \qquad \lambda = \frac{m_{new}}{m_{old}}.$$4 slow servers, then uses two routing probabilities $$\gamma_d = 2 + \frac{\lambda}{p}, \qquad \gamma_e = 1 + \frac{\lambda + p}{\lambda(1-p)}, \qquad \lambda = \frac{m_{new}}{m_{old}}.$$5 and $$\gamma_d = 2 + \frac{\lambda}{p}, \qquad \gamma_e = 1 + \frac{\lambda + p}{\lambda(1-p)}, \qquad \lambda = \frac{m_{new}}{m_{old}}.$$6 to decide whether idle slow servers should be used when sampled fast servers are busy, and whether fast or slow busy servers should be preferred when all sampled servers are busy. The large-system busy fractions satisfy

$$\gamma_d = 2 + \frac{\lambda}{p}, \qquad \gamma_e = 1 + \frac{\lambda + p}{\lambda(1-p)}, \qquad \lambda = \frac{m_{new}}{m_{old}}.$$7

and maximal stability holds for $$\gamma_d = 2 + \frac{\lambda}{p}, \qquad \gamma_e = 1 + \frac{\lambda + p}{\lambda(1-p)}, \qquad \lambda = \frac{m_{new}}{m_{old}}.$$8. Near full load, stability requires $$\gamma_d = 2 + \frac{\lambda}{p}, \qquad \gamma_e = 1 + \frac{\lambda + p}{\lambda(1-p)}, \qquad \lambda = \frac{m_{new}}{m_{old}}.$$9 and $p$0 (Gardner et al., 2020).

In distributed storage, heterogeneous scaling is realized by allowing different replicas to serialize the same logical dataset with different clustering-key orders. For a replica layout $p$1 and query $p$2, the estimated number of rows read is

$p$3

where $p$4 is the first clustering key with a range predicate. The system then chooses the replica minimizing $p$5. The Heterogeneous Replica Construction problem is to minimize average workload cost over a set of replica layouts, and the proposed HRCA solver uses simulated annealing over clustering-key permutations. On Cassandra with TPC-H data, the reported improvement in read performance reaches two orders of magnitude while maintaining similar bulk-load times; recovery from node failure remains possible, though slower because the failed layout must be rebuilt through the write path (Qiao et al., 2018).

5. Mechanical and acoustic realizations

In metamaterials, the same idea appears as scale richness. Scale-Rich networks are generated by sequentially inserting ligaments of thickness

$p$6

with random nucleation points and orientations, each grown until it meets an existing ligament or the domain boundary. This yields simultaneous heavy-tailed statistics in thickness, length, and degree. In particular,

$p$7

the degree distribution in the two-dimensional road-network representation obeys $p$8, and the length distribution has two power-law regimes with $p$9 for small $W=\{w_{ij}\}$0 and $W=\{w_{ij}\}$1 in the large-$W=\{w_{ij}\}$2 regime (Both et al., 22 Nov 2025).

The mechanical consequences are pronounced. The elastic anisotropy ratio

$W=\{w_{ij}\}$3

is tunable over a broad range at fixed density, and across 2600 distinct microstructures the authors report that Scale-Rich designs fill about $W=\{w_{ij}\}$4 of the density-anisotropy design space and achieve a 24-fold tunable range of anisotropy. Under compression, deformation remains delocalized; the localization metric stays at $W=\{w_{ij}\}$5 up to $W=\{w_{ij}\}$6 macroscopic strain. In wave control, the effective refractive index spans approximately $W=\{w_{ij}\}$7, compared with $W=\{w_{ij}\}$8 for square and hexagonal lattices over similar density ranges, enabling a Luneburg-like elastic lens assembled from scale-rich unit cells. In a payload-protection experiment, average stress on the embedded payload is reduced by up to a factor of $W=\{w_{ij}\}$9 (Both et al., 22 Nov 2025).

6. Shared mechanisms, misconceptions, and open questions

Taken together, these works suggest that HeteroScale is best understood as a principle of asymmetric resource or structure allocation across scales. In hypergraphs, the asymmetry is dual preferential attachment between nodes and hyperedges (Li et al., 2023). In weighted networks, it is the replacement of arbitrary units by an entropy-maximizing dimensionless representation (Garlaschelli et al., 2013). In LSNet, it is the separation of a large receptive field for perception from a small receptive field for aggregation (Wang et al., 29 Mar 2025). In LLM serving, it is the joint control of prefill and decode pools under heterogeneous hardware and network constraints, rather than independent autoscaling of each stage (Li et al., 27 Aug 2025). In storage, it is the use of replica diversity for latency reduction rather than mere redundancy (Qiao et al., 2018).

A common misconception is to treat HeteroScale as synonymous with ordinary multiscale processing. The cited work points to a stricter criterion: different scales or resource classes are assigned different operational roles, and those roles are coupled through an explicit control variable, routing rule, or optimization principle. This is visible in the antagonistic relation between $p(w_{ij}, z) = \frac{z w_{ij}}{1 + z w_{ij}},$0 and $p(w_{ij}, z) = \frac{z w_{ij}}{1 + z w_{ij}},$1 in double heterogeneous hypergraphs, in the single scale parameter $p(w_{ij}, z) = \frac{z w_{ij}}{1 + z w_{ij}},$2 that resolves both functional-form and unit arbitrariness in weighted networks, and in the single primary metric—decode TPS—used to coordinate both halves of a disaggregated inference pipeline (Li et al., 2023, Garlaschelli et al., 2013, Li et al., 27 Aug 2025).

The open problems are correspondingly diverse. The hypergraph model does not include deletion or rewiring and assumes linear preferential attachment (Li et al., 2023). The maximum-entropy weighted-network formalism uses a single global scale parameter and an edge-independent ensemble (Garlaschelli et al., 2013). LSNet keeps $p(w_{ij}, z) = \frac{z w_{ij}}{1 + z w_{ij}},$3 and $p(w_{ij}, z) = \frac{z w_{ij}}{1 + z w_{ij}},$4 fixed by default and does not explore larger-scale pretraining or multimodal extensions (Wang et al., 29 Mar 2025). HeteroScale for LLM inference currently relies on a fixed P/D ratio per service and assumes detailed topology and RDMA information (Li et al., 27 Aug 2025). Scale-Rich metamaterials remain primarily a two-dimensional analysis, despite a three-dimensional extension based on plates (Both et al., 22 Nov 2025). A plausible implication is that future uses of the term will increasingly converge on settings where multiple heterogeneous scales—structural, computational, or physical—must be co-designed rather than optimized independently.