Hermite ansatz for kernel eigenbasis alignment

Establish that, for every anisotropy exponent \(\alpha\geq 0\), every polynomial inner-product kernel satisfying the stated nonnegative-coefficient assumption, and every unit-norm target function \(f_\star\in L^2(p_X)\), the cumulative squared target-coefficient distribution \(A(u)\) in the true kernel eigenbasis is asymptotically equal to the corresponding distribution \(\widehat A(u)\) in the multivariate Hermite tensor basis for all \(u>0\), with relative error tending to zero as \(d\to\infty\).

Background

The paper studies kernel ridge regression with polynomial inner-product kernels under anisotropic Gaussian inputs. The kernel spectrum is asymptotically characterized, but the exact eigenfunctions of the anisotropic kernel operator are not explicitly available. This creates a difficulty in evaluating the deterministic equivalents for the bias, because those equivalents depend on the target coefficients in the true kernel eigenbasis.

The authors therefore conjecture that, for purposes of cumulative target energy below an eigenvalue threshold, the true kernel eigenbasis can be replaced by the multivariate Hermite tensor basis. Specifically, if $\theta_\beta$ denotes the target coefficient in the kernel eigenbasis and $c_\beta$ its Hermite-basis coefficient, the conjecture asserts $A(u)=(1+o_d(1))\widehat A(u)$ uniformly over positive thresholds $u$. The paper supplies numerical evidence for this claim and uses it to analyze single-index targets, but does not prove it.