Limiting behavior of sketched least squares under i.i.d. Rademacher sketches (κ4=1)

Characterize the limiting distribution and develop valid inference for the sketch-and-solve least squares estimator when the sketching matrix has i.i.d. Rademacher entries (so that the kurtosis κ_{n,4}=1), a regime in which certain projections of the estimation error are deterministically zero and the standard central limit theorem–based analysis fails.

Background

The paper establishes asymptotic normality for i.i.d. sketches under kurtosis κ{n,4}>1+δ′ and discusses why the Rademacher case κ{n,4}=1 is excluded. In that boundary case, some quadratic form variances vanish for specific directions, implying deterministic recovery of certain linear functionals of the parameter and invalidating the usual CLT-based inference.

The authors provide examples where the variance term is zero and explicitly defer a full treatment of this degenerate case, highlighting a gap in understanding the error distribution and feasible inference when κ_{n,4}=1.

References

For example, when a_n = \ta_n = (1,0,\ldots,0)\top or a_n = (1/\sqrt{2},1/\sqrt{2},0,\ldots, 0)\top and \ta_n=(1/\sqrt{2},-1/\sqrt{2},0,\ldots,0)\top, the right-hand side of iidvar is zero. Since sketches whose entries are i.i.d.~Rademacher variables are somewhat rarely used, we leave investigating this phenomenon to future work.

iidvar:

$\sigma_n^2:=\E (s_i^\top a_n \ta_n^\top s_i - a_n^\top \ta_n)^2= (a_n^\top a_n)(\ta_n^\top \ta_n)+(a_n^\top \ta_n)^2+ (\kappa_{n,4}-3)\sum_{i=1}^n (a_{n,i} \ta_{n,i})^2. $

— Inference in Randomized Least Squares and PCA via Normality of Quadratic Forms (2404.00912 - Wang et al., 1 Apr 2024) in Section 4.3 (Sketching Matrices with i.i.d. Entries), discussion following Theorem ‘Inference in Least Squares with i.i.d. Sketching’