Intrinsic-dimension empirical Bernstein inequalities for bounded self-adjoint operators

Abstract: Operator-valued concentration inequalities are foundational to the analysis of modern high-dimensional statistics and randomized algorithms. However, standard oracle bounds are frequently limited in practice: they require explicit a priori knowledge of the true variance, and often explicitly scale with the ambient dimension, rendering them vacuous for infinite-dimensional or heavily structured operators. Motivated by these challenges, we establish the first empirical Bennett and Bernstein inequalities for sums of independent, bounded, compact self-adjoint operators. Our fully data-driven bounds replace the unknown variance with an empirical estimate and rely strictly on the intrinsic dimension rather than the ambient dimension. This structural shift yields computable, dimension-free guarantees that are strictly sharper for non-isotropic random matrices and seamlessly extend to infinite-dimensional Hilbert spaces. We demonstrate that our empirical bounds achieve asymptotic sharpness with the best known oracle rates. Finally, as an independent byproduct, we derive novel empirical concentration guarantees for the intrinsic dimension itself.

• The paper develops the first empirical Bennett and Bernstein inequalities for bounded, compact self-adjoint operators, replacing ambient dimension with the intrinsic dimension tr(Σ)/‖Σ‖.
• The method estimates variance and trace parameters directly from data through differencing, master theorems, and trace-based moment control, producing computable confidence sets for operator means.
• The resulting bounds remain finite in infinite-dimensional Hilbert spaces, asymptotically match the sharp intrinsic-dimension oracle rate, and support applications including kernel PCA and functional data analysis.

The paper develops the first empirical Bennett and Bernstein concentration inequalities for sums of independent, bounded, compact self-adjoint operators on a separable Hilbert space (2605.15278). The central contribution is a shift from ambient-dimension dependencies to intrinsic-dimension dependencies — the ratio $\mathrm{tr}(\Sigma)/\|\Sigma\|$ — combined with full data-driven estimation of the variance parameters, yielding computable confidence sets that remain valid even in infinite-dimensional Hilbert spaces where ambient-dimension bounds are vacuous.

Motivation and positioning

Oracle operator-valued Bennett and Bernstein inequalities, in the intrinsic-dimension form of Martinez-Taboada and Ramdas (Martinez-Taboada et al., 13 Feb 2026), bound the tail of $S_n = \sum_{i \le n} X_i$ via

$$P\Big(\sup_{i \le n}\|S_i\| \ge r\Big) \le 2\,\frac{\mathrm{tr}(V_n)}{\sigma^2}\exp\!\Big(-\frac{n\sigma^2}{c^2}h\!\Big(\frac{cr}{n\sigma^2}\Big)\Big),$$

where $h(u) = (1+u)\log(1+u) - u$ is the Bennett rate function and $\sigma^2$ may be chosen anywhere in $[\|V_n/n\|, \mathrm{tr}(V_n/n)]$. Choosing $\sigma^2 = \|V_n\|$ makes the pre-factor exactly the intrinsic dimension $\mathrm{tr}(V_n)/\|V_n\|$, which lies between 1 and the rank, and equals $d$ only in the isotropic case. These oracle bounds, however, require exact knowledge of the variance — an assumption rarely satisfied in practice. Prior empirical remedies exist only for matrices: Wang and Ramdas' matrix empirical Bernstein inequalities (Wang et al., 2024) retain an explicit dependence on the ambient dimension $d$, making them unusable for kernel PCA or functional PCA, where operators act on infinite-dimensional RKHSs.

Technical approach

The construction follows the union-bound template of Maurer and Pontil's scalar empirical Bernstein inequality: combine an oracle tail bound with high-probability upper bounds on the unknown variance quantities. Two structural obstacles must be overcome.

Non-monotonicity of the oracle bounds. The right-hand sides of both the Bennett and Bernstein bounds are not monotone in $S_n = \sum_{i \le n} X_i$0 (the logarithmic term diverges as $S_n = \sum_{i \le n} X_i$1), suggesting that both upper and lower confidence bounds on the variance would be needed. The authors circumvent this by proving a generalized master theorem (their Theorem 3.1) showing that only a high-probability upper bound on $S_n = \sum_{i \le n} X_i$2 is strictly necessary, together with an upper bound on $S_n = \sum_{i \le n} X_i$3. This refinement is what makes the subsequent empirical construction feasible.

Halting the moment recursion. Estimating $S_n = \sum_{i \le n} X_i$4 naively by applying the oracle machinery to pairwise-differenced estimators $S_n = \sum_{i \le n} X_i$5 introduces dependence on the fourth-moment quantity $S_n = \sum_{i \le n} X_i$6; bounding $S_n = \sum_{i \le n} X_i$7 would recurse to eighth moments indefinitely. To break this loop, the authors prove a second, looser master theorem whose tail bound depends exclusively on $S_n = \sum_{i \le n} X_i$8 — never on $S_n = \sum_{i \le n} X_i$9 — obtained via a cosh-based Markov argument combined with Lieb's concavity theorem and Doob's maximal inequality over the submartingale $$P\Big(\sup_{i \le n}\|S_i\| \ge r\Big) \le 2\,\frac{\mathrm{tr}(V_n)}{\sigma^2}\exp\!\Big(-\frac{n\sigma^2}{c^2}h\!\Big(\frac{cr}{n\sigma^2}\Big)\Big),$$0. This trace-only dependence closes the recursion at the fourth moment.

Under Assumption 4.1 ($$P\Big(\sup_{i \le n}\|S_i\| \ge r\Big) \le 2\,\frac{\mathrm{tr}(V_n)}{\sigma^2}\exp\!\Big(-\frac{n\sigma^2}{c^2}h\!\Big(\frac{cr}{n\sigma^2}\Big)\Big),$$1, $$P\Big(\sup_{i \le n}\|S_i\| \ge r\Big) \le 2\,\frac{\mathrm{tr}(V_n)}{\sigma^2}\exp\!\Big(-\frac{n\sigma^2}{c^2}h\!\Big(\frac{cr}{n\sigma^2}\Big)\Big),$$2, constant mean/covariance/fourth moments), the differenced estimators satisfy $$P\Big(\sup_{i \le n}\|S_i\| \ge r\Big) \le 2\,\frac{\mathrm{tr}(V_n)}{\sigma^2}\exp\!\Big(-\frac{n\sigma^2}{c^2}h\!\Big(\frac{cr}{n\sigma^2}\Big)\Big),$$3 and $$P\Big(\sup_{i \le n}\|S_i\| \ge r\Big) \le 2\,\frac{\mathrm{tr}(V_n)}{\sigma^2}\exp\!\Big(-\frac{n\sigma^2}{c^2}h\!\Big(\frac{cr}{n\sigma^2}\Big)\Big),$$4, so scalar empirical Bernstein bounds apply to their traces, producing fully observable $$P\Big(\sup_{i \le n}\|S_i\| \ge r\Big) \le 2\,\frac{\mathrm{tr}(V_n)}{\sigma^2}\exp\!\Big(-\frac{n\sigma^2}{c^2}h\!\Big(\frac{cr}{n\sigma^2}\Big)\Big),$$5 and $$P\Big(\sup_{i \le n}\|S_i\| \ge r\Big) \le 2\,\frac{\mathrm{tr}(V_n)}{\sigma^2}\exp\!\Big(-\frac{n\sigma^2}{c^2}h\!\Big(\frac{cr}{n\sigma^2}\Big)\Big),$$6. Notably, the explicit trace bound $$P\Big(\sup_{i \le n}\|S_i\| \ge r\Big) \le 2\,\frac{\mathrm{tr}(V_n)}{\sigma^2}\exp\!\Big(-\frac{n\sigma^2}{c^2}h\!\Big(\frac{cr}{n\sigma^2}\Big)\Big),$$7 replaces the crude estimate $$P\Big(\sup_{i \le n}\|S_i\| \ge r\Big) \le 2\,\frac{\mathrm{tr}(V_n)}{\sigma^2}\exp\!\Big(-\frac{n\sigma^2}{c^2}h\!\Big(\frac{cr}{n\sigma^2}\Big)\Big),$$8 used in finite-dimensional work; it requires only Hilbert–Schmidt (square-summable eigenvalues), not trace-class, structure.

Main results

The paper delivers three principal guarantees:

• Empirical Bennett inequality (implicit rate function): with probability at least $$P\Big(\sup_{i \le n}\|S_i\| \ge r\Big) \le 2\,\frac{\mathrm{tr}(V_n)}{\sigma^2}\exp\!\Big(-\frac{n\sigma^2}{c^2}h\!\Big(\frac{cr}{n\sigma^2}\Big)\Big),$$9, $h(u) = (1+u)\log(1+u) - u$0 is controlled by the inverse of a Bennett-type function built entirely from $h(u) = (1+u)\log(1+u) - u$1 and $h(u) = (1+u)\log(1+u) - u$2.
• Empirical Bernstein inequality (closed form): the set $h(u) = (1+u)\log(1+u) - u$3 is a $h(u) = (1+u)\log(1+u) - u$4-confidence set for $h(u) = (1+u)\log(1+u) - u$5, where

$h(u) = (1+u)\log(1+u) - u$6

• Asymptotic sharpness: with error probabilities scaled as $h(u) = (1+u)\log(1+u) - u$7 and $h(u) = (1+u)\log(1+u) - u$8, the radius satisfies

$h(u) = (1+u)\log(1+u) - u$9

exactly matching the tightest known intrinsic-dimension oracle rate. Since the intrinsic oracle is strictly tighter than Tropp's ambient oracle for non-isotropic spectra, the empirical bound inherits this strict improvement — a stronger sharpness claim than the dimension-dependent sharpness established by Wang and Ramdas.

A further byproduct, derived in the appendix, is the first known empirical concentration guarantee for the intrinsic dimension itself. By coupling opposing upper/lower bounds on $\sigma^2$0 and $\sigma^2$1, the authors construct two-sided confidence intervals for $\sigma^2$2 whose width shrinks at the parametric $\sigma^2$3 rate almost surely, with an explicitly identified limiting constant involving $\sigma^2$4 and $\sigma^2$5.

Empirical validation

Two experiments support the theory. In covariance estimation with commuting $\sigma^2$6 random matrices under isotropic, completely anisotropic, and polynomially decaying spectra, the proposed operator empirical Bernstein (OEB) bound converges to the intrinsic oracle and strictly outperforms both dimension-dependent baselines (MEB 1 and MEB 2 from (Wang et al., 2024)) for sufficiently large $\sigma^2$7 in the non-isotropic regimes — crossing below the ambient oracle barrier. In kernel PCA within the infinite-dimensional RKHS induced by a Gaussian RBF kernel, ambient baselines evaluate $\sigma^2$8 and are therefore omitted as vacuous, while the OEB radius remains finite, computable via the kernel trick alone, and converges toward the intrinsic oracle across Gaussian, uniform, and exponential input distributions. Via Weyl's inequality and Davis–Kahan perturbation bounds, these operator-norm radii translate directly into finite-sample confidence sets for kernel PCA eigenvalues and eigenfunctions.

Limitations and open questions

Several restrictions are conceded explicitly. The framework requires i.i.d. observations with constant first through fourth moments, uniformly bounded operator norm, and bounded squared traces; extension to dependent structures such as operator-valued martingales is left open. The sample size must be divisible by four due to the nested differencing scheme, and the resulting effective sample sizes for variance estimation are $\sigma^2$9 and $[\|V_n/n\|, \mathrm{tr}(V_n/n)]$0, inflating constants. The secondary master theorem used to close the moment recursion is admittedly looser than the primary one, and the closed-form Bernstein bound sacrifices some tightness relative to the empirical Bennett variant, whose implicit rate function resists analytic inversion. Finally, the authors note that capturing the degree of non-commutativity within the summands — in the spirit of free-probabilistic refinements (Mehrabifard, 2023) — remains beyond the reach of the current techniques.

Conclusion

This work extends empirical Bernstein-type concentration from scalars and fixed-dimensional matrices to compact self-adjoint operators on arbitrary separable Hilbert spaces, replacing ambient dimension with intrinsic dimension throughout. The resulting bounds are fully data-driven, asymptotically match the sharpest known oracle rates, and come with novel finite-sample confidence intervals for the intrinsic dimension itself, providing a concrete toolkit for functional PCA, random graph analysis, and quantum state tomography in settings where prior guarantees were vacuous.