--- title: High-Dimensional Gaussian Single-Index Models url: https://www.emergentmind.com/topics/high-dimensional-gaussian-single-index-models type: topic --- # High-Dimensional Gaussian Single-Index Models High-dimensional Gaussian single-index models are semiparametric regression and classification models in which the response depends on a high-dimensional covariate vector only through a one-dimensional projection, while the link between that projection and the response may be unknown, nonlinear, and model-dependent. In its basic form, the model is written either as $\mathbb{E}[Y \mid X=x] = g_\star(w^\top x)$ or as $y = f(\beta^\top x,\epsilon)$ with Gaussian covariates, typically $X \sim \mathcal{N}(0,\Sigma)$ or $X \sim \mathcal{N}(0,I_d)$ [1603.03980] [1805.01820]. The topic sits at the intersection of semiparametric statistics, high-dimensional inference, structured estimation, and modern optimization: the Gaussian assumption enables Hermite expansions, Stein-type identities, SIR-based inverse regression, and sharp high-dimensional asymptotics, while sparsity, group structure, low rank, latent factors, or RKHS regularity make the model statistically and computationally tractable in regimes such as $n \ll d$ [1506.08910] [1909.03540]. ## 1. Model class and Gaussian structure The canonical high-dimensional Gaussian single-index model assumes that the response is determined by a scalar index together with noise, for example $$ Y = g_\star(w_\star^\top X) + \varepsilon $$ or, more generally, $$ y = f(\beta^\top x,\epsilon), \qquad x \sim N(0,\Sigma), $$ where the covariates are Gaussian, the index vector is high-dimensional, and the link function is either unknown or only partially specified [1603.03980] [1805.01820]. A common normalization is $\|\beta\|_2=1$ or $\beta^\top \Sigma \beta=1$, reflecting the fact that only the direction of the index is identifiable in general [1805.01820] [1511.08102]. Several papers treat the Gaussian design as the primary theoretical regime rather than as a simplifying afterthought. One formulation fixes $X_1,\dots,X_n \stackrel{\mathrm{i.i.d.}}{\sim} \mathcal{N}(0,I_d)$ and studies predictors of the form $h(x)=g(w^\top x)$ with sparse $w$ and monotone $1$-Lipschitz $g$ [1506.08910]. Another treats $x \sim \mathcal{N}(0,\Sigma)$ and focuses on testing, where the key structural quantity is $\operatorname{var}(\mathbb{E}[x\mid y])$, which is rank one under the single-index model and Gaussian design [1805.01820]. A more recent line studies proper learning under $\mathbf{x}\sim \mathcal{N}(0,I_d)$ with known link $\sigma$, emphasizing agnostic robustness and the role of the lowest nonzero Hermite coefficient of the link [2411.05708]. The Gaussian assumption is used in distinct but complementary ways. First, isotropy, such as $\mathbb{E}[x_i x_i^\top]\propto I_d$, justifies least-squares proxies for link estimation and gradient optimality conditions in alternating SIM estimators [1603.03980]. Second, Gaussianity yields orthogonality and Hermite tensorization, for example $$ \hat H_k(\mathbf{w}\cdot \mathbf{x}) = \langle \mathrm{He}_k(\mathbf{x}), \mathbf{w}^{\otimes k}\rangle, $$ which supports tensor methods, Chow-parameter methods, and gradient analyses [2411.05708]. Third, under elliptical symmetry or Gaussianity, the SIM admits a proxy linear representation $Y=X\beta+z$ with $E[X^\top z]=0$, which underlies debiased inference without estimating the unknown link [1909.03540]. This suggests that Gaussianity is not merely a distributional convenience but a source of structural identities that are unavailable, or much harder to recover, under general designs. ## 2. Identifiability, signal measures, and structural assumptions High-dimensional Gaussian SIMs are typically ill-posed without additional structure. The most common assumption is sparsity of the index, expressed as $\|w_\star\|_0 \le s$ or $\|\beta\|_0 \le s$, with $s \ll d$ or $s \ll p$ [1506.08910] [1511.08102]. More general structural constraints are encoded through atomic cardinality: $$ \| w \|_{A, 0} = \inf \left\{ \sum_a \mathds{1}[c_a > 0] \; : \; w = \sum_a c_a a, \ c_a \ge 0 \ \forall a \in A \right\}, $$ which unifies sparsity, group sparsity, and low-rank structure [1603.03980]. For unknown-link SIMs, identifiability is usually directional. In support recovery work, the normalization $\boldsymbol{\beta}_0^\top \Sigma \boldsymbol{\beta}_0 = 1$ is imposed, and the target of linearized methods becomes $c_0 \boldsymbol{\beta}_0$ with $$ \Sigma^{-1}E[Y X] = c_0 \boldsymbol{\beta}_0, $$ provided the design is Gaussian and the SIM structure holds [1511.08102]. In inference work under symmetric designs, one instead targets $\beta=\mu\tau$, where $$ \mu := E[y_n \langle x_n,\tau\rangle] = E[f(\zeta)\zeta],\quad \zeta\sim N(0,1), $$ and interprets $\beta$ as the vector of average partial effects [1909.03540]. A central signal-strength concept for Gaussian SIMs is the generalized signal-to-noise ratio, $$ \lambda := \text{the unique non-zero eigenvalue of } \operatorname{var}(\mathbb{E}[x \mid y]), $$ defined when $\operatorname{var}(\mathbb{E}[x \mid y])$ is non-vanishing [1805.01820]. In linear Gaussian regression this $\lambda$ is equivalent, up to constants under bounded-spectrum assumptions, to the classical SNR; in general SIMs it remains identifiable even though $\|\beta\|_2^2$ is not [1805.01820]. For learning with known link under Gaussian design, another signal quantity is the information exponent $$ k^\ast := \min\{ k \ge 0 : c_k \neq 0\}, $$ the degree of the first nonzero Hermite coefficient in the expansion of the link [2411.05708]. A related scalar in sparse Gaussian SIM learning is $$ \theta := \mathbb{E}_{\mu \sim \mathcal{N}(0,1)}[g_\star(\mu)\mu], $$ which appears in excess-risk bounds and measures correlation between the transformed signal and the latent Gaussian index [1506.08910]. These notions clarify a common misconception: Gaussian SIMs are not identified solely through the coefficient vector. Depending on the problem, the meaningful signal may be the direction of $\beta$, the rank-one inverse-regression spike $\lambda$, the linear projection constant $c_0$, or the first informative Hermite level $k^\ast$ [1805.01820] [2411.05708]. A plausible implication is that different tasks—estimation, support recovery, testing, or robust proper learning—probe different facets of the same underlying one-dimensional structure. ## 3. Estimation methods for the unknown index and link A major line of work addresses simultaneous estimation of the index and link under Gaussian design. In sparse Gaussian SIMs with monotone $1$-Lipschitz link, the papers on SILO, iSILO, and ciSILO begin from the model $$ \mathbb{E}[Y \mid X=x] = g_\star(w_\star^\top x), $$ with $X \sim \mathcal{N}(0,I_d)$, $Y\in[0,1]$, and sparse $w_\star$ [1506.08910]. SILO first estimates $w$ through a lasso-style convex program independent of $g$, then estimates the link via LPAV, the Lipschitz Pool Adjacent Violators procedure. LPAV solves $$ \hat{z} = \arg\min_{z\in\mathbb{R}^n} \|z - y\|_2^2 \quad \text{s.t.}\quad 0 \le z_j - z_i \le p_j - p_i \quad \text{if } p_i \le p_j, $$ with $p_i=\hat w^\top x_i$, and extends the solution by linear interpolation [1506.08910]. The iterative variants then alternate between updating the index and re-estimating the link, either with squared loss or with a calibrated loss whose derivative equals the current link estimate [1506.08910]. The structurally constrained CSI framework generalizes this idea to sparsity, group sparsity, and low-rank models. It approximately solves $$ \hat{g},\hat{w} = \arg\min_{g\in G,w}\ \frac{1}{n}\sum_{i=1}^n \Phi(w^\top x_i) - y_i w^\top x_i + \frac{\lambda}{2}\|w\|_2^2 \quad \text{s.t. } \|w\|_{A,0} \le s, \ \Phi' = g, $$ where $G$ is the set of monotone, $1$-Lipschitz functions [1603.03980]. Each iteration estimates $g$ by LPAV and updates $w$ by a gradient step followed by atomic projection. In the sparse case the projection reduces to hard thresholding; in the low-rank case it becomes best rank-$s$ approximation [1603.03980]. The Gaussian role is explicit in the LPAV derivation: under $\mathbb{E}[x_i x_i^\top]\propto I_d$, as in $X\sim \mathcal{N}(0,\sigma^2 I_d)$, the least-squares link fit aligns with the first-order condition for the calibrated loss [1603.03980]. Another route dispenses with explicit link estimation when the objective is support recovery or inference on the linearized parameter. Under Gaussian design, ordinary least-squares LASSO applied to the misspecified linear model can still recover the signed support of the true SIM index because the population least-squares target is proportional to the true direction [1511.08102]. The estimator is the standard $$ \widehat{\boldsymbol{\beta} = \arg\min_{\boldsymbol{\beta}\in\mathbb{R}^p} \left\{ \frac{1}{2n}\sum_{i=1}^n (Y_i - \mathbf{X}_i^\top \boldsymbol{\beta})^2 + \lambda \|\boldsymbol{\beta}\|_1 \right\}, $$ and under Gaussian $\mathbf{X}$, irrepresentability, and bounded-spectrum assumptions, it achieves exact signed support recovery at the optimal rescaled sample size $n_{p,s}=n/[s\log(p-s)]$ up to constants [1511.08102]. For inference, debiased Lasso estimators under symmetric or Gaussian designs achieve root-$n$ consistency and asymptotic normality while bypassing estimation of the unknown link [1909.03540]. These approaches illustrate a division of labor. When prediction and link estimation are central, Gaussian isotropy motivates alternating schemes such as CSI and ciSILO [1603.03980] [1506.08910]. When support recovery or coordinate-wise inference is central, Gaussian structure makes it possible to reduce the SIM to a proxy linear problem and use sparse linear tools directly [1511.08102] [1909.03540]. ## 4. Detection, support recovery, and inferential theory The testing problem in Gaussian SIMs is sharply characterized in terms of the generalized signal-to-noise ratio $\lambda$. For sparse alternatives with Gaussian design, the null is $$ H_0:\beta_1=\cdots=\beta_p=0,\ \text{or equivalently gSNR}=0, $$ against sparse nonzero alternatives [1805.01820]. Under the high-dimensional scaling $$ s^{2}\log^2(p)\wedge p \prec n, $$ the paper establishes minimax detection boundaries. For the general SIM $y=f(\beta^\top x,\epsilon)$, detection is possible if and only if $$ \lambda \succ \frac{s\log(p)}{n}\wedge\frac{p^{1/2}}{n}, $$ while for the additive SIM $y=f(\beta^\top x)+\epsilon$, detection is possible if and only if $$ \lambda \succ \frac{s\log(p)}{n}\wedge\frac{p^{1/2}}{n}\wedge\frac{1}{\sqrt{n}} $$ [1805.01820]. The additive-noise boundary matches that of high-dimensional linear regression, a result explicitly described as surprising in the source material [1805.01820]. The testing procedures achieving these rates are based on Sliced Inverse Regression. After slicing the response into $H$ slices and forming the SIR estimator $$ \widehat{\Lambda}_H = \frac{1}{H}\sum_{h=1}^{H}\bar{x}_{h,\cdot}\bar{x}_{h,\cdot}^{\tau}, $$ the test combines a global largest-eigenvalue statistic $\lambda_{\max}(\widehat{\Lambda}_H)$ and a sparse largest-eigenvalue statistic $\lambda^{(ks)}_{\max}(\widehat{\Lambda}_H)$, with a Gaussian multiplier or semidefinite relaxation used for computational tractability [1805.01820]. This testing theory emphasizes a distinction sometimes blurred in the literature: in sparse Gaussian SIMs one can often detect a signal at substantially weaker strength than is required to estimate the direction consistently [1805.01820]. Support recovery theory gives a parallel threshold phenomenon. In Gaussian SIMs with sparse $\beta_0$, support recovery is impossible below a critical effective sample size $$ n_{p,s} := \frac{n}{s \log(p-s)}, $$ and, for a broad class of SIMs with Gaussian design, if $n_{p,s}