Semiparametric Efficiency Theory

• Semiparametric efficiency theory is a framework that defines the minimal asymptotic variance for estimators in models with both finite-dimensional and infinite-dimensional components.
• It leverages geometrical concepts in Hilbert spaces, using tangent space projections to construct efficient scores, influence functions, and establish information bounds.
• Applications in partially linear additive models demonstrate practical efficiency gains through smooth backfitting and one-step correction, enhancing estimator performance under regularity conditions.

Semiparametric efficiency theory provides a rigorous framework for characterizing the minimal asymptotic variance achievable by any regular estimator in models with both finite-dimensional (parametric) and infinite-dimensional (nonparametric) components. Central to this theory is the geometric structure of the underlying Hilbert space of score functions, the construction and projection of tangent spaces, and the explicit derivation of efficient scores, influence functions, and information bounds. The analysis in the context of partially linear additive models—where additive structure is imposed on the nonparametric component—reveals both conceptual and practical efficiency gains when exploiting structural information in the nuisance part. This paradigm fundamentally shapes modern approaches to semi-parametric inference and estimator construction.

1. Model Structure and Regularity

The canonical partially linear additive model for semiparametric efficiency theory is

$Y = X^\top\beta + m(Z) + \epsilon,$

with $X\in\mathbb{R}^p$, $Z\in[0,1]^d$, $m(Z)=\sum_{j=1}^d m_j(Z_j)$, and $\epsilon$ independent. Regularity and identifiability are enforced by

• Centering: $E[m_j(Z_j)] = 0$ for all $j$,
• Smoothness: each $m_j$ is $C^2$,
• Joint density $q_{X,Z}$ bounded and bounded away from zero,
• $E\|X\|^r < \infty$ for some $r > 2$,
• $\epsilon$ absolutely continuous, symmetric, with density $g$ satisfying $\int (g')^2/g < \infty$ and $E[\epsilon^2] < \infty$.

The Hilbert space $\mathcal{H}$ of additive functions (zero-centered, square integrable under $Z$) is defined to formalize the geometric tangent space for the nonparametric component.

2. Tangent Spaces and Efficient Score Construction

Let $S_{\text{full}}$ denote the score for the full model under differentiable submodels in both the parametric and nonparametric directions,

$$S_{\rm full} = \varphi(\epsilon)X + \varphi(\epsilon)\delta(Z), \quad \varphi(u) = \frac{g'(u)}{g(u)},$$

where $\delta\in\mathcal{H}$ is a tangent direction in $m$. The nuisance tangent space consists of all elements of the form $\varphi(\epsilon)\delta(Z)$.

Efficient score construction then proceeds by projecting $S_{\rm full}$ orthogonally onto the complement of the nuisance tangent space with respect to the $L_2(P)$ inner product. The least-favorable direction $$\delta^*(z) = -\Pi\left(E[X\mid Z=z]\,|\,\mathcal{H}\right)$$ is obtained by solving

$$E\left[\left(X+\delta^*(Z)\right)\delta(Z)\right] = 0 \quad \forall \delta\in\mathcal{H}.$$

Let $$\eta_j(z) = \Pi\left(E[X_j\mid Z = z] \mid \mathcal{H}\right)$$ and $\eta = (\eta_1, ..., \eta_p)$. The resulting efficient score is

$$S_{\rm eff}(Y,X,Z;\beta) = \varphi\left(Y - X^\top\beta - m(Z)\right)\left[X - \eta(Z)\right],$$

and, at the truth $(\beta^0, m^0)$,

$$S_{\rm eff} = \frac{g'}{g}\Bigl(\epsilon\Bigr)\left[X - \eta(Z)\right], \quad \epsilon = Y - X^\top\beta^0 - m^0(Z).$$

3. Semiparametric Fisher Information Bound and Influence Function

The semiparametric Fisher information matrix is given by

$$I_{\rm eff}(\beta^0) = E\left[S_{\rm eff}S_{\rm eff}^\top\right] = I_g\,E\left[(X - \eta(Z))(X - \eta(Z))^\top\right],$$

where $I_g = \int (g')^2/g$. The Cramér–Rao lower bound asserts that, for any regular estimator $\hat\beta$, $$\operatorname{Var}(\hat\beta) \succeq \frac{1}{n}I_{\rm eff}(\beta^0)^{-1}$$.

The efficient influence function attaining this bound is

$$\psi(Y,X,Z) = I_{\rm eff}(\beta^0)^{-1} \varphi\left(Y - X^\top\beta^0 - m^0(Z)\right)\, [X - \eta(Z)],$$

with $E[\psi] = 0$ and $E[\psi\psi^\top] = I_{\rm eff}^{-1}$.

4. Construction of Semiparametrically Efficient Estimators

Efficient estimation is achieved via a two-step ("profile plus one-step") procedure:

• Step A: Construct the Gaussian-profile estimator:
  1. For candidate $\beta$, regress $R_i(\beta) = Y_i - X_i^\top\beta$ on $Z$ additively using smooth backfitting, obtaining $\hat m_j(z_j;\beta)$, form $\hat m(z;\beta) = \sum_j \hat m_j(z_j;\beta)$.
  2. Fit $\hat\eta(z)$ by backfitting all $X_j$ on $Z$.
  3. Define $\widetilde Y^i = Y^i - \hat m(Z^i;\beta)$ and $\widetilde X^i = X^i - \hat\eta(Z^i)$.
  4. Obtain the estimator:

  $$\hat\beta = \left(\sum_i \widetilde X^i\widetilde X^{i\top}\right)^{-1} \sum_i \widetilde X^i\,\widetilde Y^i.$$

  Under Gaussian noise, $$\sqrt n(\hat\beta - \beta^0) \xrightarrow[]{d} N(0, \sigma^2 M^{-1})$$, $M = E[(X - \eta(Z))(X - \eta(Z))^\top]$, which is only semiparametrically efficient if $g$ is Gaussian.

• Step B: Apply a one-step adaptation to correct for non-Gaussian $g$:
  1. Compute residuals $$\hat\epsilon^i = Y^i - X^{i\top}\hat\beta - \hat m(Z^i;\hat\beta)$$.
  2. Estimate $\varphi(u)=g'(u)/g(u)$ via kernel-density estimation on the residuals (exploiting symmetry).
  3. Update, forming the estimated information:

  $$\hat I_{\rm eff} = \left[\frac{1}{n}\sum_i \widetilde X^i\widetilde X^{i\top}\right]\,\left[\frac{1}{n}\sum_i \hat\varphi(\hat\epsilon^i)^2\right].$$

4. Define the one-step estimator:

$$\tilde\beta = \hat\beta - \hat I_{\rm eff}^{-1}\frac{1}{n}\sum_{i=1}^n \widetilde X^i\,\hat\varphi(\hat\epsilon^i).$$

Under standard regularity and estimation conditions, one attains

$$\sqrt n(\tilde\beta - \beta^0) \xrightarrow[]{d} N(0, I_{\rm eff}(\beta^0)^{-1}),$$

ensuring semiparametric efficiency.

5. The Impact of Nonparametric Structure and Efficiency Gains

The essential insight is that modeling the nonparametric component $m(Z)$ with additive structure (as opposed to completely nonparametric $m$) facilitates a strictly smaller nuisance tangent space $\mathcal{H}$, and thus the projection defining the efficient score is less aggressive. The quantity $E[X\mid Z]$ admits an additive structure-compliant $L_2$-projection, making estimation of the parametric component $\beta$ more efficient. Consequently, the information matrix $I_{\rm eff}$ is strictly larger (i.e., the bound is lower) than in the unrestricted partially linear model when $E[X\mid Z]$ is non-additive, as the additive assumption successfully removes nonidentifiable nuisance directions.

Simulation results demonstrate that the smooth-backfitted Gaussian-profile estimator ("SAM") outperforms classical profile-kernel estimators for the partially linear model, often by large mean-squared-error factors for complex $E[X|Z]$ structure. The adaptive one-step ("ASAM") estimator achieves additional efficiency gains when error distributions are non-Gaussian, empirically reducing MSE and respecting the theoretical lower bound.

6. Practical Implementation, Regularity, and Limitations

Implementation requires smooth additive regression (e.g., via smooth backfitting), kernel density-derivative estimation for $\varphi$, and precise centering of $m_j(Z_j)$. All algorithms admit computationally tractable forms for moderate dimensions $d$ (alleviating the curse of dimensionality). Key regularity assumptions include:

• Twice differentiability of $m_j$,
• Boundedness and positivity of joint densities,
• Symmetry and sufficient smoothness of $g$,
• Independence of $\epsilon$ from $(X,Z)$.

Performance may degrade for high $d$ due to the quality of additive approximations and the kernel estimation step. However, the overall framework is robust and generalizes to partial linear models with further structured nonparametric components.

7. Broader Context and Applications

This framework generalizes the Bickel–Klaassen–Ritov–Wellner approach for semiparametric models, emphasizing the construction of the tangent space for the precise nonparametric structure imposed. Efficient influence functions and estimation procedures, including smooth backfitting and sample-splitting for $\eta(Z)$, are central regardless of the statistical model, and have informed much subsequent work on double machine learning and structured semiparametric regression. When applied to real data (e.g., Boston housing), the proposed method not only fits well but also correctly flags cases where non-Gaussian residual structure is present, thereby providing more reliable inference on covariate effects.