Metric Complexity: Theory & Applications

• Metric Complexity is a family of quantitative constructs that capture nuanced structural, geometric, and dynamic properties rather than relying on simple size-based measures.
• It finds applications in diverse areas such as neural network memorization diagnostics, symbolic regression via Hessian rank, and entropy-based invariants in compact metric spaces.
• Empirical studies show its effectiveness in diagnosing overfitting and aligning model expressiveness with data curvature, while also revealing limitations in scalability and layer sensitivity.

Searching arXiv for papers on “metric complexity” and closely related uses of the term across domains. Metric complexity denotes a family of quantitative constructs rather than a single canonical invariant. In the cited literature, the term is used for empirical surrogates of model capacity in deep learning, data-informed post hoc selectors for symbolic regression, invariants of compact metric spaces, ordinal-valued decomposition complexity in coarse geometry, descriptive complexity classes of geometric distances, query complexity phenomena in metric optimization, graph-theoretic resolving-set parameters, and domain-specific software or physical complexity measures. Across these settings, a common pattern is the replacement of coarse size-based notions by metrics that depend on geometry, data, dynamics, or task structure (Becker et al., 2024, Haut et al., 29 Jan 2025, Aishwarya et al., 13 Jul 2025).

1. Neural-network memorization as a complexity metric

In supervised deep learning, metric complexity has been defined through the ability of a model to fit random labels in parallel with true labels. A multi-head network augments a standard CNN feature extractor with two heads running in parallel: a true-label head producing a softmax over the actual classes and a random-label head producing, for each class $j$, a softmax over $n$ randomly assigned labels. At training time one reads out only the random-label prediction $\hat p^y$ corresponding to the true class $y$ (Becker et al., 2024).

For a training set $S=\{(x_i,y_i)\}_{i=1}^m$ with i.i.d. random labels $s_i\sim \mathrm{Uniform}(\{1,\dots,n\})$, the memorization score is

$$\mathrm{Mem}_S(H)=\frac1m\sum_{i=1}^m \mathbf{1}\!\left\{\arg\max_k \hat p_i^{y_i}[k]=s_i\right\}.$$

This score is used as an empirical surrogate for the Rademacher complexity. The paper recalls the binary bound

$$\hat{\mathfrak R}_m(H)=\mathbb{E}_\sigma\Big[\sup_{h\in H}\frac1m\sum_{i=1}^m \sigma_i h(x_i)\Big],$$

and the accompanying generalization inequality

$$R(h)\le \hat R_S(h)+\mathfrak R_m(H)+\sqrt{\frac{\log(1/\delta)}{2m}},$$

then interprets fitting the random labels $s_i$ as aligning with a random label assignment, so that high $n$0 high capacity (Becker et al., 2024).

The method is presented as model-agnostic, not relying on parameter counts, norms, or architecture details, and as directly observing actual SGD-trained behavior on noise rather than using quantities such as VC-dimension or norm-based bounds. Its stated limitations are that it scales poorly to very large $n$1 and measures memorization at the layer or layers where heads attach, so it cannot detect memorization moved into earlier layers (Becker et al., 2024).

Experimentally, on WideResNet-16-4 trained on CIFAR100 with SGD, momentum $n$2, cosine decay, batch size $n$3, 200 epochs, and default $n$4 random labels, the random-label accuracy climbs from $n$5 to nearly $n$6 when $n$7, even after true-label train accuracy approaches $n$8, indicating continued overfitting. Dropout, weight decay, and label smoothing each reduce $n$9. A copy-depth study on VGG16 shows that memorization “turns on” deep in the convolutional stack, specifically layers 5–7. The proposed random-label regularizer suppresses random-label accuracy, yet test accuracy on CIFAR100 does not improve and often degrades, contrary to classical bounds predicting that lower $\hat p^y$0 should improve generalization (Becker et al., 2024).

This suggests that, in this setting, metric complexity is effective as a diagnostic of memorization and comparative capacity, but not as a direct prescription for improving out-of-sample performance.

2. Data-informed model complexity in symbolic regression

In symbolic regression, a different metric complexity is defined through Hessian rank. For a twice-differentiable regression model $\hat p^y$1, the Hessian at $\hat p^y$2 is

$\hat p^y$3

The procedure evaluates the Hessian at three strategic points $\hat p^y$4, chosen in practice as points with minimal, mean, and maximal target values in the training data, forms the average Hessian

$\hat p^y$5

and defines the effective dimensionality

$\hat p^y$6

namely the number of eigenvalues of $\hat p^y$7 whose magnitude exceeds a numerical threshold $\hat p^y$8 (Haut et al., 29 Jan 2025).

In the reported implementation, second derivatives are approximated by centered finite differences and rank is computed via an SVD with threshold $\hat p^y$9. The runtime is stated as $y$0 per model. This complexity metric is then aligned with data complexity estimated by intrinsic dimensionality. The study uses twelve estimators from the scikit-dimension library: CorrInt, DANCo, ESS, FS, KNN, lPCA, MADA, MiND ML, MLE, MoM, TLE, and TwoNN, then computes

$y$1

The data-informed selection rule is

$y$2

and selects models satisfying $y$3 (Haut et al., 29 Jan 2025).

On 121 symbolic-regression PMLB problems using StackGP, the paper reports that models whose $y$4 aligns with data intrinsic dimension generalize best. The normalized test error summary is reported as follows (Haut et al., 29 Jan 2025):

Group	# models	median normalized error ± SE
Ideal	420	0.010 ± 0.013
Close (±1)	380	0.021 ± 0.015
Far	500	0.057 ± 0.016

Pairwise Mann–Whitney tests give $y$5 for all comparisons. The paper further states a strong U-shape in $y$6 versus normalized test error, with minimum error attained when $y$7 (Haut et al., 29 Jan 2025).

The theoretical intuition links Hessian rank to expressiveness on a smooth $y$8-dimensional manifold $y$9. If $S=\{(x_i,y_i)\}_{i=1}^m$0 varies only along $S=\{(x_i,y_i)\}_{i=1}^m$1, then $S=\{(x_i,y_i)\}_{i=1}^m$2 and $S=\{(x_i,y_i)\}_{i=1}^m$3; under nondegeneracy, $S=\{(x_i,y_i)\}_{i=1}^m$4. Averaging over points in $S=\{(x_i,y_i)\}_{i=1}^m$5 yields $S=\{(x_i,y_i)\}_{i=1}^m$6 under general position assumptions (Haut et al., 29 Jan 2025).

3. Metric complexity as an invariant of compact metric spaces

In metric geometry, metric complexity is an isometry-invariant of compact metric spaces that generalizes cardinality. For a finite subset $S=\{(x_i,y_i)\}_{i=1}^m$7 at scale $S=\{(x_i,y_i)\}_{i=1}^m$8, one forms the Laplace-kernel similarity

$S=\{(x_i,y_i)\}_{i=1}^m$9

and for a probability vector $s_i\sim \mathrm{Uniform}(\{1,\dots,n\})$0 defines

$s_i\sim \mathrm{Uniform}(\{1,\dots,n\})$1

The complexity of the finite metric space $s_i\sim \mathrm{Uniform}(\{1,\dots,n\})$2 is

$s_i\sim \mathrm{Uniform}(\{1,\dots,n\})$3

and for compact $s_i\sim \mathrm{Uniform}(\{1,\dots,n\})$4,

$s_i\sim \mathrm{Uniform}(\{1,\dots,n\})$5

In the discrete metric case, $s_i\sim \mathrm{Uniform}(\{1,\dots,n\})$6, so $s_i\sim \mathrm{Uniform}(\{1,\dots,n\})$7 (Aishwarya et al., 13 Jul 2025).

The invariant is monotone under inclusion, contracts under $s_i\sim \mathrm{Uniform}(\{1,\dots,n\})$8-Lipschitz maps, and is therefore preserved by isometries. Replacing $s_i\sim \mathrm{Uniform}(\{1,\dots,n\})$9 by $$\mathrm{Mem}_S(H)=\frac1m\sum_{i=1}^m \mathbf{1}\!\left\{\arg\max_k \hat p_i^{y_i}[k]=s_i\right\}.$$0 scales the parameter reciprocally: $$\mathrm{Mem}_S(H)=\frac1m\sum_{i=1}^m \mathbf{1}\!\left\{\arg\max_k \hat p_i^{y_i}[k]=s_i\right\}.$$1 The cited paper establishes a connection with Bryant–Tupper diversities by showing that

$$\mathrm{Mem}_S(H)=\frac1m\sum_{i=1}^m \mathbf{1}\!\left\{\arg\max_k \hat p_i^{y_i}[k]=s_i\right\}.$$2

is a diversity on finite subsets. The key wedge-sum inequality is

$$\mathrm{Mem}_S(H)=\frac1m\sum_{i=1}^m \mathbf{1}\!\left\{\arg\max_k \hat p_i^{y_i}[k]=s_i\right\}.$$3

for pointed finite metric spaces $$\mathrm{Mem}_S(H)=\frac1m\sum_{i=1}^m \mathbf{1}\!\left\{\arg\max_k \hat p_i^{y_i}[k]=s_i\right\}.$$4 and $$\mathrm{Mem}_S(H)=\frac1m\sum_{i=1}^m \mathbf{1}\!\left\{\arg\max_k \hat p_i^{y_i}[k]=s_i\right\}.$$5 glued at the distinguished point (Aishwarya et al., 13 Jul 2025).

On compact subsets of $$\mathrm{Mem}_S(H)=\frac1m\sum_{i=1}^m \mathbf{1}\!\left\{\arg\max_k \hat p_i^{y_i}[k]=s_i\right\}.$$6, the induced diversity is Minkowski-superlinear: $$\mathrm{Mem}_S(H)=\frac1m\sum_{i=1}^m \mathbf{1}\!\left\{\arg\max_k \hat p_i^{y_i}[k]=s_i\right\}.$$7 The paper also gives explicit examples. For two points at distance $$\mathrm{Mem}_S(H)=\frac1m\sum_{i=1}^m \mathbf{1}\!\left\{\arg\max_k \hat p_i^{y_i}[k]=s_i\right\}.$$8,

$$\mathrm{Mem}_S(H)=\frac1m\sum_{i=1}^m \mathbf{1}\!\left\{\arg\max_k \hat p_i^{y_i}[k]=s_i\right\}.$$9

This use of metric complexity is explicitly described as a metric-sensitive analogue of maximum entropy (Aishwarya et al., 13 Jul 2025).

A distinct but related geometric usage studies the descriptive-set-theoretic complexity of standard distances such as Gromov–Hausdorff, Banach–Mazur, Kadets, Lipschitz, Net, and Hausdorff–Lipschitz distance. These are shown to be mutually Borel-uniformly-continuous bi-reducible and therefore to share the same descriptive-set-theoretic complexity (Cúth et al., 2020). Although the paper concerns the complexity of distances rather than the invariant $$\hat{\mathfrak R}_m(H)=\mathbb{E}_\sigma\Big[\sup_{h\in H}\frac1m\sum_{i=1}^m \sigma_i h(x_i)\Big],$$0, it places metric-sensitive quantities within a common complexity framework.

4. Ordinal, query, and graph-theoretic notions

A further mathematical use of the term appears in coarse geometry through ordinal-valued decomposition complexity. For a metric family $$\hat{\mathfrak R}_m(H)=\mathbb{E}_\sigma\Big[\sup_{h\in H}\frac1m\sum_{i=1}^m \sigma_i h(x_i)\Big],$$1, one defines classes $$\hat{\mathfrak R}_m(H)=\mathbb{E}_\sigma\Big[\sup_{h\in H}\frac1m\sum_{i=1}^m \sigma_i h(x_i)\Big],$$2 by transfinite induction: $$\hat{\mathfrak R}_m(H)=\mathbb{E}_\sigma\Big[\sup_{h\in H}\frac1m\sum_{i=1}^m \sigma_i h(x_i)\Big],$$3 consists of all uniformly bounded families, and for $$\hat{\mathfrak R}_m(H)=\mathbb{E}_\sigma\Big[\sup_{h\in H}\frac1m\sum_{i=1}^m \sigma_i h(x_i)\Big],$$4,

$$\hat{\mathfrak R}_m(H)=\mathbb{E}_\sigma\Big[\sup_{h\in H}\frac1m\sum_{i=1}^m \sigma_i h(x_i)\Big],$$5

A space has complexity exactly $$\hat{\mathfrak R}_m(H)=\mathbb{E}_\sigma\Big[\sup_{h\in H}\frac1m\sum_{i=1}^m \sigma_i h(x_i)\Big],$$6 if it belongs to $$\hat{\mathfrak R}_m(H)=\mathbb{E}_\sigma\Big[\sup_{h\in H}\frac1m\sum_{i=1}^m \sigma_i h(x_i)\Big],$$7 but to no earlier $$\hat{\mathfrak R}_m(H)=\mathbb{E}_\sigma\Big[\sup_{h\in H}\frac1m\sum_{i=1}^m \sigma_i h(x_i)\Big],$$8. The paper gives five equivalent characterizations of complexity $$\hat{\mathfrak R}_m(H)=\mathbb{E}_\sigma\Big[\sup_{h\in H}\frac1m\sum_{i=1}^m \sigma_i h(x_i)\Big],$$9, proves that $$R(h)\le \hat R_S(h)+\mathfrak R_m(H)+\sqrt{\frac{\log(1/\delta)}{2m}},$$0 has exact complexity $$R(h)\le \hat R_S(h)+\mathfrak R_m(H)+\sqrt{\frac{\log(1/\delta)}{2m}},$$1, and that $$R(h)\le \hat R_S(h)+\mathfrak R_m(H)+\sqrt{\frac{\log(1/\delta)}{2m}},$$2 has complexity exactly $$R(h)\le \hat R_S(h)+\mathfrak R_m(H)+\sqrt{\frac{\log(1/\delta)}{2m}},$$3 (Zhu et al., 2016).

In query complexity, the metric Steiner Tree problem exhibits a sharp contrast with metric MST. Given an unknown metric $$R(h)\le \hat R_S(h)+\mathfrak R_m(H)+\sqrt{\frac{\log(1/\delta)}{2m}},$$4 on $$R(h)\le \hat R_S(h)+\mathfrak R_m(H)+\sqrt{\frac{\log(1/\delta)}{2m}},$$5 points and a terminal set $$R(h)\le \hat R_S(h)+\mathfrak R_m(H)+\sqrt{\frac{\log(1/\delta)}{2m}},$$6 of size $$R(h)\le \hat R_S(h)+\mathfrak R_m(H)+\sqrt{\frac{\log(1/\delta)}{2m}},$$7, the goal is to estimate

$$R(h)\le \hat R_S(h)+\mathfrak R_m(H)+\sqrt{\frac{\log(1/\delta)}{2m}},$$8

The paper proves that any randomized algorithm estimating Steiner tree cost within factor $$R(h)\le \hat R_S(h)+\mathfrak R_m(H)+\sqrt{\frac{\log(1/\delta)}{2m}},$$9 requires $s_i$0 queries, while a sublinear-query algorithm achieves a $s_i$1-approximation with $s_i$2 queries, and any better-than-$s_i$3 estimate requires $s_i$4 queries (Chen et al., 2022). The authors explicitly describe these results as illustrating a “phase diagram” for metric query complexity.

Graph theory provides yet another family of metric complexity notions. For a connected graph $s_i$5, the metric dimension is the minimum size of a resolving set $s_i$6 such that every pair $s_i$7 is resolved by some $s_i$8, meaning $s_i$9. The problem is NP-complete on planar graphs of maximum degree $n$00, but solvable in polynomial time on outerplanar graphs by a dynamic program on a generalized dual tree (Diaz et al., 2011). A generalized version is the $n$01-metric dimension: a set $n$02 is a $n$03-metric generator if each pair of distinct vertices is distinguished by at least $n$04 vertices in $n$05. Computing the $n$06-metric dimension is NP-complete in general, though linear-time algorithms are available for trees (Yero et al., 2014).

These usages are not interchangeable. They share a dependence on distances or decompositions, but one addresses coarse ordinal invariants, another oracle-query lower bounds, and another resolving-set optimization in finite graphs.

5. Dataset, code, and algorithmic cost metrics

In image classification, dataset complexity has been quantified by the cumulative spectral gradient (CSG). Starting from class-overlap estimates

$n$07

one forms a symmetric affinity matrix

$n$08

the graph Laplacian $n$09, and its eigenvalues $n$10. The normalized eigengaps are

$n$11

and the complexity score is

$n$12

The paper reports Pearson correlations between CSG and CNN test error on six 10-class sets of $n$13 for AlexNet, $n$14 for ResNet-50, and $n$15 for Xception, with CSG described as more accurate and faster than previous complexity measures (Branchaud-Charron et al., 2019).

In software engineering, complexity metrics are tailored to artifacts rather than abstract spaces. For test code, CCTR is defined at class or suite granularity as

$n$16

with $n$17, hence

$n$18

Here $n$19 is SonarSource-style control-flow nesting complexity, $n$20 counts assertion-style or fail statements, $n$21 counts mocking-related invocation patterns, and $n$22 scores annotation roles, with simple annotations contributing $n$23 and rich or parameterized annotations contributing $n$24 (Ouédraogo et al., 7 Jun 2025). The study evaluates 15,750 test suites generated by EvoSuite, GPT-4o, and Mistral Large-1024 across 350 classes from Defects4J and SF110, reporting that CCTR discriminates between structured and fragmented suites and that Kolmogorov–Smirnov tests show $n$25 for EvoSuite versus GPT-4o distributions (Ouédraogo et al., 7 Jun 2025).

At class level, Complete Class Complexity (CCC) is the unweighted sum

$n$26

where the terms count methods, average cyclomatic complexity, aggregation, external calls, superclasses, immediate subclasses, implemented interfaces, imported packages, and return points (Singh et al., 2014). The authors evaluate CCC via Weyuker’s properties and state that it satisfies all except property 9, while property 7 is inapplicable to object-oriented programs (Singh et al., 2014).

For algorithm analysis, a recent multi-metric formulation explicitly generalizes asymptotic unit-cost models. Let $n$27 be a set of instruction classes and assign each class $n$28 a cost vector

$n$29

If a code artifact $n$30 has instruction counts $n$31, raw metric totals are

$n$32

normalized cohort-wise by min–max scaling, and aggregated with a user-supplied weight vector $n$33 on the $n$34-simplex into

$n$35

The paper presents this as “Metric Complexity” beyond asymptotic analysis, with dimensions computational effort, energy usage, carbon footprint, and monetary cost, and reports strong correlations $n$36 with measured data across architectures (Kavun, 18 Aug 2025).

6. Physical and optimization-theoretic interpretations

In disordered photonic crystals, modal complexity is defined from the complex-valued wavefunction $n$37. The squared complexity is

$n$38

or in discretized form,

$n$39

The paper proposes the average value and statistical distribution of $n$40 as a metric for Anderson localization, exploiting openness of the disordered medium and allowing determination of localization length (Mondal et al., 2024).

The same study models $n$41 statistically via an $n$42-distribution when the real and imaginary parts of the field are Gaussian. If the mean complexity $n$43, then

$n$44

with fit parameters $n$45 and $n$46. The mean complexity is connected to the dimensionless conductance $n$47 and localization length $n$48 through

$n$49

and, in the limit $n$50,

$n$51

(Mondal et al., 2024).

In AdS/CFT, metric complexity can be associated to the Bures metric on density matrices. For $n$52, the fidelity and Bures distance are

$n$53

For a one-parameter family $n$54, the infinitesimal cost is $n$55, and the Bures complexity is

$n$56

subject to the prescribed channel dynamics. The paper computes this measure for reduced density matrices on single intervals in descendant states of the vacuum in 2d CFTs and derives a bulk dual observable localized in the entanglement wedge (Gerbershagen et al., 2024).

Nonconvex low-rank optimization furnishes another meaning. For the rank-one generalized matrix completion problem

$n$57

the instance $n$58 is assigned a distance-to-ambiguity

$n$59

where $n$60 is the set of ambiguous instances, and the complexity metric is

$n$61

Small $n$62 means far from ambiguous, while large $n$63 means close to ambiguous. The paper proves a sufficient condition for absence of spurious local minima when $n$64, and a necessary condition for their existence, under additional assumptions, when $n$65 (zhang et al., 2022).

A final asymmetry-based interpretation appears in the complexity quasi-metric

$n$66

on running-time functions $n$67. Here $n$68 iff $n$69 for all $n$70, so “fast $n$71 slow” costs zero but “slow $n$72 fast” costs $n$73. The scaling transformation $n$74 satisfies

$n$75

and is expansive iff $n$76. The associated $n$77-stable sets recover pointwise complexity classes such as the cone of functions at least as slow as $n$78 (Gaba, 7 Feb 2026).

7. Common themes and recurring limitations

Across these literatures, metric complexity is repeatedly used to replace coarse proxies by measurements tied to actual structure. In neural networks, the metric observes actual SGD-trained behavior on noise rather than parameter counts or norms (Becker et al., 2024). In symbolic regression, it aligns model curvature directions with dataset intrinsic dimensionality rather than relying on parsimony pressure or size alone (Haut et al., 29 Jan 2025). In compact metric spaces, it generalizes cardinality through a Laplace-kernel entropy maximization that is sensitive to interpoint distances (Aishwarya et al., 13 Jul 2025). In software and algorithm analysis, it augments control-flow or asymptotic counts with assertions, annotations, mocking, energy, carbon, and monetary dimensions (Ouédraogo et al., 7 Jun 2025, Kavun, 18 Aug 2025).

At the same time, each formulation comes with explicit scope conditions. The random-label memorization score assumes an expressive network and may miss memorization shifted to earlier layers (Becker et al., 2024). The Hessian-rank metric depends on finite-difference approximations, thresholding, and post-processing rather than in-loop optimization (Haut et al., 29 Jan 2025). CCTR and CCC are artifact-specific and intentionally reflect software conventions rather than general mathematical complexity (Ouédraogo et al., 7 Jun 2025, Singh et al., 2014). The weighted-operation model depends on calibrated cost tables, microbenchmarking, and hardware versioning (Kavun, 18 Aug 2025). The modal-complexity criterion requires phase-sensitive measurements (Mondal et al., 2024). The distance-to-ambiguity metric for matrix completion is conceptually unifying, but the paper notes that computing $n$79 requires solving a non-convex optimization over nearby ambiguous instances (zhang et al., 2022).

A plausible implication is that “metric complexity” functions less as a single theory than as a methodological stance: complexity is measured through an explicitly chosen metric structure that reflects the phenomenon of interest, whether memorization, curvature, interpoint similarity, query access, software readability, physical openness, or optimization landscape geometry.