Dice Question Streamline Icon: https://streamlinehq.com

Limiting behavior of sketched least squares under i.i.d. Rademacher sketches (κ4=1)

Characterize the limiting distribution and develop valid inference for the sketch-and-solve least squares estimator when the sketching matrix has i.i.d. Rademacher entries (so that the kurtosis κ_{n,4}=1), a regime in which certain projections of the estimation error are deterministically zero and the standard central limit theorem–based analysis fails.

Information Square Streamline Icon: https://streamlinehq.com

Background

The paper establishes asymptotic normality for i.i.d. sketches under kurtosis κ{n,4}>1+δ′ and discusses why the Rademacher case κ{n,4}=1 is excluded. In that boundary case, some quadratic form variances vanish for specific directions, implying deterministic recovery of certain linear functionals of the parameter and invalidating the usual CLT-based inference.

The authors provide examples where the variance term is zero and explicitly defer a full treatment of this degenerate case, highlighting a gap in understanding the error distribution and feasible inference when κ_{n,4}=1.

References

For example, when a_n = \ta_n = (1,0,\ldots,0)\top or a_n = (1/\sqrt{2},1/\sqrt{2},0,\ldots, 0)\top and \ta_n=(1/\sqrt{2},-1/\sqrt{2},0,\ldots,0)\top, the right-hand side of iidvar is zero. Since sketches whose entries are i.i.d.~Rademacher variables are somewhat rarely used, we leave investigating this phenomenon to future work.

iidvar:

$\sigma_n^2:=\E (s_i^\top a_n \ta_n^\top s_i - a_n^\top \ta_n)^2= (a_n^\top a_n)(\ta_n^\top \ta_n)+(a_n^\top \ta_n)^2+ (\kappa_{n,4}-3)\sum_{i=1}^n (a_{n,i} \ta_{n,i})^2. $

— Inference in Randomized Least Squares and PCA via Normality of Quadratic Forms (2404.00912 - Wang et al., 1 Apr 2024) in Section 4.3 (Sketching Matrices with i.i.d. Entries), discussion following Theorem ‘Inference in Least Squares with i.i.d. Sketching’