Improved Variance Estimation in Homoskedastic Nonparametric Random-Design Regression via a Two-Scale Approach

Abstract: We study estimation of a constant conditional variance $σ<sup>2$ in nonparametric regression with a $d$-dimensional random design. This is an important problem, and similar questions arise in causal inference. The regression function is $β_b$-Hölder smooth, the design density is $β_g$-Hölder smooth and bounded above and away from zero, and we consider the nonparametric regime $β_b&gt;1$ and $d&gt;4β_b$. Set $$β_g<sup>\star=β_b(1-4β_b/d)/{1+2β_b/d+8(β_b/d)<sup>2}$$. We give an estimator whose mean squared error is upper bounded by $Cn<sup>{-4(β_b+1)/(d+4)}$ in the low-regularity regime when $0&lt;β_g\leqβ_g<sup>\star$. The low-regularity branch is based on a new two-scale construction: the covariate space is partitioned into cells, the local polynomial trend is projected out within each suitable cell, and the squared normalized contrast from one eligible close pair per cell is averaged across cells. In the high regularity regime when $β_g&gt;β_g<sup>\star$, a higher-order influence function estimator of Robins, Li, Tchetgen Tchetgen, and van der Vaart (2008) provides the rate $Cn<sup>{-8β_b/(d+4β_b)}$. We also give an all-pairs ridge extension, which achieves the same two-scale rate, and evaluate the methods alongside a range of existing estimators in simulations.