Effective Span Dimension: Signal-Aware Complexity

• Effective Span Dimension is an alignment-sensitive complexity measure defined as the smallest number of eigen-directions required for the tail signal energy to fall below the noise level.
• It balances bias and variance in spectral estimation, offering minimax risk guarantees and improved performance in over-parameterized settings.
• Extensions of ESD apply to fixed-design regression and RKHS methods, enabling adaptive learning across diverse kernel frameworks.

Searching arXiv for papers on “Effective Span Dimension” and closely related “effective dimension” work. Effective Span Dimension (ESD) is an alignment-sensitive complexity measure for spectral algorithms with learned kernels. In the formulation introduced in "Alignment-Sensitive Minimax Rates for Spectral Algorithms with Learned Kernels" (Huang et al., 24 Sep 2025), ESD depends jointly on the target signal, the eigenvalue ordering or spectrum, and the noise level $\sigma^2$. Its operative meaning is the smallest truncation level at which the average remaining tail signal energy falls below the noise level. This makes ESD a signal-aware alternative to spectrum-only complexity measures, and the framework is stated to be well-defined for arbitrary kernels and signals without requiring eigen-decay conditions or source conditions (Huang et al., 24 Sep 2025).

1. Formal definition

The basic construction is given in a sequence model

$z_j=\theta_j^*+\xi_j,\qquad j=1,\dots,d,$

where the noises $\xi_j$ are uncorrelated, mean-zero, and have variance $\sigma^2$. Let the eigenvalues be sorted in descending order as $\lambda_{\pi_1}>\lambda_{\pi_2}>\cdots$. The associated trade-off function is

$$H_{\bm{\theta}^*,\bm{\lambda}}(k)=\frac{1}{k}\sum_{i=k+1}^{d}\theta_{\pi_i^*}^{2}, \qquad k\in[d].$$

The Effective Span Dimension is then defined by

$$d^\dagger = d^\dagger(\sigma^2;\bm{\theta}^*,\bm{\lambda}) = \min\left\{k\in[d]: \frac{1}{k}\sum_{i=k+1}^{d}\theta_{\pi_i^*}^{2}\le \sigma^2\right\}.$$

Equivalently, $d^\dagger$ is the smallest number of leading eigen-directions needed so that the average remaining tail energy falls below the noise level. The paper also defines the span profile

$$D_{\bm{\theta}^*,\bm{\lambda}}(\tau)=d^\dagger(\tau;\bm{\theta}^*,\bm{\lambda}),$$

which records how ESD changes as the noise level varies (Huang et al., 24 Sep 2025).

This definition is explicitly alignment-sensitive. Two spectra with identical eigenvalue sets can have different ESDs if the target signal is arranged differently relative to the ordered eigen-directions. The framework is presented as a response to the limitation of classical kernel complexity measures that depend only on the spectrum and not on signal-kernel alignment (Huang et al., 24 Sep 2025).

2. Bias-variance interpretation and minimax consequences

The paper develops the intuition through principal component truncation. If the first $k$ coordinates are retained, the bias is

$z_j=\theta_j^*+\xi_j,\qquad j=1,\dots,d,$0

while the variance is $z_j=\theta_j^*+\xi_j,\qquad j=1,\dots,d,$1. In this formulation, ESD identifies the point at which the bias-variance balance becomes favorable. The optimal principal component risk is bounded by

$z_j=\theta_j^*+\xi_j,\qquad j=1,\dots,d,$2

Accordingly, once ESD is at most $z_j=\theta_j^*+\xi_j,\qquad j=1,\dots,d,$3, the best possible risk among principal-component estimators is of order $z_j=\theta_j^*+\xi_j,\qquad j=1,\dots,d,$4 (Huang et al., 24 Sep 2025).

The same paper states a minimax characterization. For the class

$z_j=\theta_j^*+\xi_j,\qquad j=1,\dots,d,$5

the minimax excess risk satisfies

$z_j=\theta_j^*+\xi_j,\qquad j=1,\dots,d,$6

In the asymptotic regime with $z_j=\theta_j^*+\xi_j,\qquad j=1,\dots,d,$7, this becomes a rate of order $z_j=\theta_j^*+\xi_j,\qquad j=1,\dots,d,$8, and the optimally tuned principal-component estimator matches the lower bound up to constants (Huang et al., 24 Sep 2025).

The importance of these statements is methodological rather than merely definitional. ESD is not introduced only as a descriptive statistic; it is used to index function classes and to determine minimax rates. In that sense, it plays the role of an alignment-sensitive statistical complexity parameter for spectral estimation.

3. Extensions beyond the sequence model

The same framework is extended to fixed-design linear regression by whitening and applying the singular value decomposition

$z_j=\theta_j^*+\xi_j,\qquad j=1,\dots,d,$9

The transformed signal and spectrum are

$\xi_j$0

and ESD becomes

$\xi_j$1

For principal component regression, the optimal tuned prediction risk satisfies

$\xi_j$2

and the minimax prediction risk over the class $\xi_j$3 is

$\xi_j$4

The linear model is therefore presented not as a separate theory, but as the same ESD principle after reduction to a sequence model (Huang et al., 24 Sep 2025).

For RKHS regression, the paper uses the Mercer decomposition

$\xi_j$5

Because random design introduces extra variance, the effective per-coordinate noise is taken to be

$\xi_j$6

The ESD is then

$\xi_j$7

For the kernel principal component projection estimator, the optimal tuned risk obeys

$\xi_j$8

Under a mild bounded-target assumption and a regularity condition on the kernel spectrum, the minimax risk over the corresponding span-profile class is

$\xi_j$9

These extensions are part of the paper’s claim that ESD applies to arbitrary kernels and arbitrary signals without source conditions or eigen-decay assumptions (Huang et al., 24 Sep 2025).

4. Learned kernels and over-parameterized gradient flow

A central motivation for ESD is the analysis of learned or adaptive spectra. In the sequence-model treatment of over-parameterized gradient flow, each coordinate is parameterized as

$\sigma^2$0

and the learned eigenvalues at time $\sigma^2$1 are

$\sigma^2$2

The resulting pathwise ESD is

$\sigma^2$3

Under regularity conditions on signal strengths and initialization, the paper states that for sufficiently late times,

$\sigma^2$4

The interpretation given is that stronger signal coordinates cause their associated learned eigenvalues to grow faster, making the ordering of the learned spectrum more favorable to the signal and decreasing the tail energy beyond the leading learned directions (Huang et al., 24 Sep 2025).

The analysis is supported by conservation laws,

$\sigma^2$5

which are used to show that strong-signal coordinates acquire larger effective eigenvalues. The article also reports numerical examples in synthetic sequence models, fixed-design linear regression, and RKHS regression. In those experiments, the span profile worsens as a misalignment parameter increases, then shifts downward during training; oracle PCR risk varies in lockstep with ESD; and traditional spectrum-only measures do not distinguish cases with identical eigenvalue sets but different signal alignment (Huang et al., 24 Sep 2025).

This places ESD within a broader program of explaining why learned kernels can improve generalization. The core claim is not that feature learning helps in an undifferentiated way, but that adaptive training can reduce the number of leading directions effectively required for estimation.

5. Relation to other effective-dimension frameworks

The phrase "Effective Span Dimension" is specific to the learned-kernel setting above, but it belongs to a broader landscape of finite-sample and information-theoretic dimension notions. A distinct line of work defines a scale-dependent effective dimension for statistical models using the Fisher Information Matrix as a Riemannian metric and a covering-number viewpoint at statistical resolution $\sigma^2$6. In that setting,

$\sigma^2$7

so directions with Fisher eigenvalues much smaller than $\sigma^2$8 contribute little at sample size $\sigma^2$9 (Berezniuk et al., 2020).

A separate Bayesian formulation defines the effective dimension through mutual information:

$\lambda_{\pi_1}>\lambda_{\pi_2}>\cdots$0

with

$\lambda_{\pi_1}>\lambda_{\pi_2}>\cdots$1

In regular parametric models this converges to the ambient parameter dimension, while in ill-posed or strongly regularized settings it can be substantially smaller (Banerjee, 28 Dec 2025).

There is also an algorithmic-fractal notion of effective Hausdorff dimension,

$\lambda_{\pi_1}>\lambda_{\pi_2}>\cdots$2

which is used to analyze pinned distance sets and to connect Kolmogorov complexity to Hausdorff dimension via the point-to-set principle (Stull, 2022). Another recent development studies gauge profiles of the sets

$\lambda_{\pi_1}>\lambda_{\pi_2}>\cdots$3

again in the sense of effective dimension rather than span-sensitive spectral complexity (Miao, 31 Jan 2026).

These notions are not interchangeable. The Fisher-geometric definition is scale-dependent and model-geometric; the Bayesian definition is mutual-information based and coordinate-free; the algorithmic definitions are Kolmogorov-complexity based; and Effective Span Dimension, in the strict sense of (Huang et al., 24 Sep 2025), is alignment-sensitive and constructed from the joint behavior of signal, ordered spectrum, and noise.

6. Terminological ambiguity of “ESD”

The acronym ESD is heavily overloaded across fields. In the literature represented here, only one paper uses it to mean Effective Span Dimension (Huang et al., 24 Sep 2025). Several other meanings are current:

ESD meaning	Domain	Paper
Electrostatic Discharge	LVDS protection and termination	(Gaddipati, 22 May 2025)
Enhanced Span-based Decomposition	Few-shot sequence labeling	(Wang et al., 2021)
Empirical spectral distribution	Random matrix theory	(Vargas, 2017)
Expected Squared Difference	Neural calibration	(Yoon et al., 2023)
Erroneous Span Detection	GEC and MT evaluation	(Chen et al., 2020, Lyu et al., 13 Mar 2026)

This ambiguity is not merely lexical. In (Gaddipati, 22 May 2025), ESD refers to Electrostatic Discharge protection in high-speed LVDS systems, with quantitative guidance on clamping voltages, capacitance constraints, termination, and layout. In (Wang et al., 2021), ESD denotes Enhanced Span-based Decomposition, a span-level few-shot sequence labeling framework built around enhanced span representation, class prototype aggregation, and span conflict resolution. In (Vargas, 2017), ESD means empirical spectral distribution, specifically the mean ESD of the squared unimodular random matrix. In (Yoon et al., 2023), ESD is Expected Squared Difference, a trainable calibration objective. In (Chen et al., 2020) and (Lyu et al., 13 Mar 2026), ESD stands for Erroneous Span Detection or Error Span Detection in grammatical error correction and machine translation evaluation.

For this reason, the phrase Effective Span Dimension should generally be written in full on first use, especially in cross-disciplinary settings. A plausible implication is that acronym-only citation of "ESD" is unusually prone to category error unless the surrounding mathematical context makes the intended meaning explicit.