On the Structural Dimension of Sliced Inverse Regression

Abstract: In this work, we address the longstanding puzzle that Sliced Inverse Regression (SIR) often performs poorly for sufficient dimension reduction when the structural dimension $d$ (the dimension of the central space) exceeds 4. We first show that in the multiple index model $Y=f( \mathbf{P} \boldsymbol{X})+\epsilon$ where $\boldsymbol{X}$ is a $p$-standard normal vector, $\epsilon$ is an independent noise, and $\mathbf{P}$ is a projection operator from $\mathbb R^{p}$ to $\mathbb R^{d}$, if the link function $f$ follows the law of a Gaussian process, then with high probability, the $d$-th eigenvalue $\lambda_{d}$ of $\mathrm{Cov}\left[\mathbb{E}(\boldsymbol{X}\mid Y)\right]$ satisfies $\lambda_{d}\leq C e^{-\theta d}$ for some positive constants $C$ and $\theta$. We then focus on the low signal regime where $\lambda_{d}$ can be arbitrarily small and not larger than $d^{-8.1}$, and prove that the minimax risk of estimating the central space is lower bounded by $\frac{dp}{n\lambda_{d}}$. Combining these two results, we provide a convincing explanation for the poor performance of SIR when $d$ is large, a phenomenon that has perplexed researchers for nearly three decades. The technical tools developed here may be of independent interest for studying other sufficient dimension reduction methods.