Dimension-Free Generalization Bounds

• Dimension-free generalization bounds are upper limits on the generalization gap that remove explicit dependence on ambient dimensions by relying on intrinsic measures like norm constraints, compression rates, and algorithm-dependent geometry.
• They employ mechanisms such as norm-preserving linearization, hypothesis localization, and compression techniques to control complexity without depending on architectural details like depth, width, or parameter count.
• These bounds offer practical insights in contexts like supervised quantum machine learning and nonlinear metric learning, ensuring consistent convergence rates (e.g., 1/√n) despite increasing model size.

Dimension-free generalization bounds are upper bounds on the generalization gap $R(h)-\widehat R_S(h)$ that avoid explicit dependence on ambient input dimension, parameter count, width, depth, or analogous architectural size measures, and instead depend on sample size together with intrinsic quantities such as norm constraints, observable scale, compression rate, or algorithm-dependent geometric complexity. In the standard learning-theoretic sense made explicit for supervised quantum machine learning, a bound is dimension-free when it contains no explicit dependence on the Hilbert-space dimension $2^N$, the Pauli-vector dimension $4^N$, or explicit capacity surrogates such as VC dimension, covering number, or parameter count (Wang et al., 28 Oct 2025). Recent arXiv work shows that this notion now spans several distinct regimes: strictly architecture-free uniform convergence, algorithm-dependent bounds indexed by reachable hypothesis sets, rate-distortion and compression formulations, data-dependent fractal dimensions, finite-space concentration on digital domains, and width-free nonlinear metric-learning bounds (Wang et al., 28 Oct 2025, Tan et al., 2022, Sachs et al., 2023, Sefidgaran et al., 2022, Kratsios et al., 2024, Dupuis et al., 2023, Kozdoba et al., 2021).

1. Learning-theoretic meaning

The common object of study is the difference between a population quantity and its empirical counterpart. In supervised quantum machine learning this is written as

$\operatorname{gen}(h_S)=R(h_S)-\widehat R_S(h_S),$

where $R(h_S)$ is the prediction risk and $\widehat R_S(h_S)$ is the empirical risk on the training sample $S$ (Wang et al., 28 Oct 2025). In algorithm-dependent Rademacher analyses the same object appears as $R(\hat\theta)-\hat R(\hat\theta,S^n)$, and the formal distinction is not the target quantity but the complexity notion used to control it (Sachs et al., 2023).

A strict dimension-free bound removes ambient dimension from the bound itself. The strongest examples make the leading term depend only on sample size and a small set of normalized quantities. A broader usage, increasingly common in modern deep-learning theory, treats a bound as dimension-free when ambient dimension disappears but is replaced by an intrinsic quantity: a Hausdorff dimension of the SGD-reachable set, a finite Minkowski dimension of an algorithm-dependent class, a rate-distortion function of the output distribution, or a loss-space box dimension of a random hypothesis set (Tan et al., 2022, Sachs et al., 2023, Sefidgaran et al., 2022, Dupuis et al., 2023).

This distinction matters because “dimension-free” is not synonymous with “complexity-free.” Several papers explicitly separate ambient-dimension independence from dependence on intrinsic geometry, compression, stability, or norm parameters. The resulting literature is best read as a taxonomy of ways to replace crude size measures by quantities that are either normalized, algorithm-dependent, or data-dependent.

2. Structural mechanisms for removing ambient dimension

One recurring mechanism is norm-preserving linearization. In supervised quantum machine learning, a parameterized circuit followed by measurement can be rewritten in a Pauli basis as a linear predictor $$h(\boldsymbol{x})=\boldsymbol{w}^\top\boldsymbol{x}$$ with $\|\boldsymbol{w}\|_2=1$ and $2^N$0; the Rademacher complexity then scales as $2^N$1 without explicit dependence on the Pauli-vector dimension $2^N$2 because the orthogonal Pauli transfer matrix preserves norms (Wang et al., 28 Oct 2025). The same pattern reappears in nonlinear metric learning: once the metric is normalized by $2^N$3 and each coordinate has bounded amplification, averaging over coordinates prevents the complexity from growing with width $2^N$4 (Kozdoba et al., 2021).

A second mechanism is localization to the hypothesis set actually explored by the algorithm. In the fractal-SGD approach, the bound depends on the Hausdorff dimension $2^N$5 of the SGD-reachable set $2^N$6, not on the ambient parameter dimension $2^N$7 (Tan et al., 2022). In the ARC framework, the relevant class is

$2^N$8

and generalization is controlled by the empirical Rademacher complexity of that algorithm- and data-dependent class rather than the full parameter space (Sachs et al., 2023).

A third mechanism is compression. Rate-distortion formulations replace dimension by coding complexity: the expected generalization error can be bounded in terms of an algorithmic rate $2^N$9, and under Lipschitz control of the loss this is upper bounded by the classical rate-distortion function $4^N$0 of the output distribution $4^N$1 (Sefidgaran et al., 2022). Related finite-space work on digital computation replaces ambient Euclidean dimension by a geometric representation dimension $4^N$2 of a bi-Lipschitz embedding of the finite learning problem into $4^N$3 (Kratsios et al., 2024).

A fourth mechanism is to define geometry directly in loss space. Data-dependent fractal bounds dispense with a parameter-space Lipschitz assumption by introducing the pseudo-metric

$4^N$4

and then measuring the upper box-counting dimension of the hypothesis set under $4^N$5 (Dupuis et al., 2023). This makes the complexity depend on how many loss-distinguishable hypotheses exist for the observed sample, rather than on how many free parameters the model has.

3. Supervised quantum machine learning as a strict dimension-free case

The most explicit recent strict dimension-free result is the high-probability uniform bound for supervised quantum machine learning in “Tight Generalization Bound for Supervised Quantum Machine Learning” (Wang et al., 28 Oct 2025). The setting uses an i.i.d. training set

$4^N$6

where each $4^N$7 is an $4^N$8-qubit density matrix and the model output is

$4^N$9

After Pauli expansion, the model becomes

$\operatorname{gen}(h_S)=R(h_S)-\widehat R_S(h_S),$0

with $\operatorname{gen}(h_S)=R(h_S)-\widehat R_S(h_S),$1, $\operatorname{gen}(h_S)=R(h_S)-\widehat R_S(h_S),$2, and $\operatorname{gen}(h_S)=R(h_S)-\widehat R_S(h_S),$3. For any non-negative risk function bounded by $\operatorname{gen}(h_S)=R(h_S)-\widehat R_S(h_S),$4 and $\operatorname{gen}(h_S)=R(h_S)-\widehat R_S(h_S),$5-Lipschitz in its first argument, the paper proves that with probability at least $\operatorname{gen}(h_S)=R(h_S)-\widehat R_S(h_S),$6,

$\operatorname{gen}(h_S)=R(h_S)-\widehat R_S(h_S),$7

This bound is explicit, uniformly valid over the hypothesis class, and free of big-$\operatorname{gen}(h_S)=R(h_S)-\widehat R_S(h_S),$8 notation (Wang et al., 28 Oct 2025).

What appears in the bound is sharply limited: the sample size $\operatorname{gen}(h_S)=R(h_S)-\widehat R_S(h_S),$9, the spectral norm $R(h_S)$0 of the measurement observable, the Lipschitz constant $R(h_S)$1, the bounded-loss range $R(h_S)$2, and the confidence parameter $R(h_S)$3. What does not appear is equally significant: the number of qubits $R(h_S)$4, the Pauli-vector dimension $R(h_S)$5, circuit depth, number of parameterized gates, total parameter count, encoding scheme, learning rate, batch size, number of epochs, and optimizer. The paper stresses that this independence is formal rather than heuristic: it follows from orthogonality of the Pauli transfer matrix and from the norm/output constraints used in the Rademacher calculation (Wang et al., 28 Oct 2025).

The same work also emphasizes empirical tightness. Across classification and regression tasks, the empirical generalization error decreases at approximately the same $R(h_S)$6 pace as the bound; varying the number of parameterized gates from $R(h_S)$7 to $R(h_S)$8, the number of qubits from $R(h_S)$9 to $\widehat R_S(h_S)$0, and training hyperparameters leaves generalization essentially unchanged; and the upper bound remains valid even when labels are completely randomized (Wang et al., 28 Oct 2025). Within its stated regime—Pauli-string observables, bounded losses, and no data re-uploading—the result is not merely ambient-dimension-free but also width-free, depth-free, and parameter-count-free.

4. Algorithm-dependent and fractal formulations

A different line of work retains dimension-free form by replacing ambient dimension with the geometry of optimization trajectories. “Learning Non-Vacuous Generalization Bounds from Optimization” models SGD by a stochastic differential equation driven by fractional Brownian motion and defines $\widehat R_S(h_S)$1 as the union of SGD trajectories across datasets and mini-batch schedules. The main high-probability theorem bounds $\widehat R_S(h_S)$2 in terms of the Hausdorff dimension $\widehat R_S(h_S)$3, the diameter $\widehat R_S(h_S)$4 of the set of achievable loss vectors, the sample size $\widehat R_S(h_S)$5, and a Lipschitz constant $\widehat R_S(h_S)$6, with no explicit dependence on the ambient parameter dimension $\widehat R_S(h_S)$7, width, or depth (Tan et al., 2022). The complexity term is intrinsic and algorithm-dependent rather than structural or worst-case.

“Generalization Guarantees via Algorithm-dependent Rademacher Complexity” abstracts that idea into a general symmetrization framework. Its central object is the algorithm- and data-dependent class $\widehat R_S(h_S)$8, and the key technical lemma gives

$\widehat R_S(h_S)$9

together with a high-probability analogue controlled by the essential supremum of the same ARC quantity (Sachs et al., 2023). This perspective yields a dimension-independent bound for strongly convex, smooth SGD, finite Minkowski-dimension bounds for finite reachable classes, and recoveries of compression-scheme and VC-style results. The paper is explicit that these are ambient-dimension-free but not strictly complexity-free: the replacement terms are contraction parameters, covering numbers, or intrinsic fractal dimensions.

“Generalization Bounds with Data-dependent Fractal Dimensions” removes a different bottleneck: the global Lipschitz-in-parameters assumption that earlier fractal bounds required. It introduces the loss-based pseudo-metric $S$0 and controls the generalization gap by the upper box-counting dimension $S$1 of a fixed or random hypothesis set under that pseudo-metric (Dupuis et al., 2023). For random hypothesis spaces, the bound also contains total mutual information terms such as $S$2, and the paper introduces “geometric stability” to connect local covering geometry to global information-theoretic dependence. The result is ambient-dimension-free, but it is explicitly data-dependent and algorithm-dependent.

These three frameworks share a common move: ambient dimension is discarded only after the hypothesis class has been localized. The localization may be dynamical, as in reachable SGD trajectories; combinatorial, as in $S$3; or loss-geometric, as in $S$4-box dimension. The complexity that remains is therefore intrinsic to what the training procedure actually explores.

5. Compression, finite-space, and metric-learning perspectives

Several other frameworks replace dimension by compressibility, representation dimension, or bounded per-coordinate amplification.

Framework	Complexity term	Dimension statement
Rate-distortion learning	$S$5, $S$6, $S$7	No explicit ambient dimension in the main bounds
Finite-space digital learning	$S$8, $S$9, $R(\hat\theta)-\hat R(\hat\theta,S^n)$0	Rate exponent independent of ambient $R(\hat\theta)-\hat R(\hat\theta,S^n)$1
Nonlinear metric learning	$R(\hat\theta)-\hat R(\hat\theta,S^n)$2 or $R(\hat\theta)-\hat R(\hat\theta,S^n)$3	No width $R(\hat\theta)-\hat R(\hat\theta,S^n)$4, and only logarithmic dependence on $R(\hat\theta)-\hat R(\hat\theta,S^n)$5

In the rate-distortion formulation, expected generalization obeys

$R(\hat\theta)-\hat R(\hat\theta,S^n)$6

and under a Lipschitz condition on the loss one may upper-bound $R(\hat\theta)-\hat R(\hat\theta,S^n)$7 by the classical rate-distortion function $R(\hat\theta)-\hat R(\hat\theta,S^n)$8 (Sefidgaran et al., 2022). The lossless case recovers the mutual-information bound of Xu and Raginsky. The lossy case is more flexible: it trades exact reconstruction of the learned hypothesis for approximation in generalization performance. The same paper also connects asymptotic generalization to the rate-distortion dimension $R(\hat\theta)-\hat R(\hat\theta,S^n)$9, thereby making “dimension” an information-geometric property of the output distribution rather than a parameter count (Sefidgaran et al., 2022).

“Tighter Learning Guarantees on Digital Computers via Concentration of Measure on Finite Spaces” takes a different route. Once learning is modeled on finite metric spaces $$h(\boldsymbol{x})=\boldsymbol{w}^\top\boldsymbol{x}$$0 and $$h(\boldsymbol{x})=\boldsymbol{w}^\top\boldsymbol{x}$$1, with a bi-Lipschitz embedding $$h(\boldsymbol{x})=\boldsymbol{w}^\top\boldsymbol{x}$$2, the uniform generalization gap over a Lipschitz hypothesis class is controlled by a bound whose rate factor is $$h(\boldsymbol{x})=\boldsymbol{w}^\top\boldsymbol{x}$$3, yielding the family $$h(\boldsymbol{x})=\boldsymbol{w}^\top\boldsymbol{x}$$4 advertised in the abstract (Kratsios et al., 2024). The paper is explicit that this is dimension-adaptive rather than absolutely dimension-free: the ambient Euclidean dimension $$h(\boldsymbol{x})=\boldsymbol{w}^\top\boldsymbol{x}$$5 disappears from the rate exponent, but the representation dimension $$h(\boldsymbol{x})=\boldsymbol{w}^\top\boldsymbol{x}$$6, the distortion $$h(\boldsymbol{x})=\boldsymbol{w}^\top\boldsymbol{x}$$7, and the finite cardinality $$h(\boldsymbol{x})=\boldsymbol{w}^\top\boldsymbol{x}$$8 remain.

“Dimension Free Generalization Bounds for Non Linear Metric Learning” gives yet another model-specific route. For a one-layer embedding $$h(\boldsymbol{x})=\boldsymbol{w}^\top\boldsymbol{x}$$9, the sparse regime assumes $\|\boldsymbol{w}\|_2=1$0 and $\|\boldsymbol{w}\|_2=1$1, leading to

$\|\boldsymbol{w}\|_2=1$2

More strikingly, the bounded-amplification regime assumes only $\|\boldsymbol{w}\|_2=1$3, and because the metric is normalized by $\|\boldsymbol{w}\|_2=1$4 and decomposes as an average over coordinates, the paper proves

$\|\boldsymbol{w}\|_2=1$5

with no dependence on width $\|\boldsymbol{w}\|_2=1$6 and only logarithmic dependence on input dimension $\|\boldsymbol{w}\|_2=1$7 (Kozdoba et al., 2021). This is a concrete example where architecture growth is neutralized by averaging structure rather than sparsity.

6. Hidden dependencies, limitations, and recurring misconceptions

The first recurrent misconception is that dimension-free means universally insensitive to model size. Recent work does not support that reading. The quantum bound is formally independent of qubit count, circuit depth, and optimizer choice, but it still depends on the observable norm $\|\boldsymbol{w}\|_2=1$8, the loss range $\|\boldsymbol{w}\|_2=1$9, and the Lipschitz constant $2^N$00; if $2^N$01 or the loss scale grows with system size, implicit dimension dependence reappears (Wang et al., 28 Oct 2025). Rate-distortion bounds have no explicit $2^N$02, yet $2^N$03 and $2^N$04 can scale with ambient dimension for unstructured output distributions (Sefidgaran et al., 2022). Digital finite-space bounds suppress ambient $2^N$05 but still depend on the representation dimension $2^N$06, the distortion $2^N$07, and the cardinality $2^N$08 (Kratsios et al., 2024).

A second misconception is that all dimension-free bounds are assumption-light. The opposite is often true. The Hausdorff-dimension SGD theory relies on an FBM-driven SDE approximation, bounded drift and diffusion, countable support, and Ahlfors regularity of the reachable set $2^N$09 (Tan et al., 2022). Data-dependent fractal bounds require bounded continuous loss and, for the cleaner information-theoretic form, geometric stability of coverings in loss space (Dupuis et al., 2023). The strict QML result excludes data re-uploading architectures and is specialized to observables measured in expectation, with bounded losses and Pauli-decomposition structure (Wang et al., 28 Oct 2025). These are mathematically coherent regimes, but they are not universal.

A third issue is vacuity hidden by asymptotics. One of the strongest methodological claims in the recent QML literature is that big-$2^N$10 notation can conceal constants so large that ostensibly benign bounds become larger than the maximal possible generalization error on realistic sample sizes (Wang et al., 28 Oct 2025). This criticism extends beyond quantum learning: several modern papers are motivated by the gap between asymptotic, architecture-sensitive complexity bounds and the empirically non-vacuous behavior of overparameterized models. The move toward explicit constants, algorithm-dependent hypothesis classes, and intrinsic geometry is partly a response to that problem.

The resulting picture is therefore layered. Some bounds are genuinely architecture-free once natural normalization conditions are imposed, as in the quantum linearization and bounded-amplification metric-learning results. Others are better described as ambient-dimension-free but intrinsic-dimension-controlled, because they replace width or parameter count by Hausdorff dimension, Minkowski dimension, compression rate, mutual information, or representation dimension. Across these settings, the leading statistical scale remains sample-size driven—typically $2^N$11, or finite-space variants thereof—while model dependence survives only through normalized or intrinsic quantities. That is the operative modern meaning of dimension-free generalization bounds.