--- title: Efficient Orthogonal Moments in Semiparametric Models url: https://www.emergentmind.com/topics/semiparametrically-efficient-orthogonal-moments type: topic --- # Efficient Orthogonal Moments in Semiparametric Models Semiparametrically efficient orthogonal moments are moment functions, scores, or estimating equations for a finite-dimensional target parameter that satisfy two properties simultaneously: they are Neyman-orthogonal to nuisance perturbations, so their population expectation is first-order insensitive to errors in the nuisance, and they coincide with, or induce, the efficient score or efficient influence function, so that regular estimators based on them attain the semiparametric variance lower bound. In modern usage, the concept spans conditional moment restriction models, high-dimensional regularized M- and Z-estimation, double/debiased machine learning, multivariate elicitable functionals, and generated-regressor settings with latent effects or fixed effects [1806.04823] [2010.14146] [2303.11418]. ## 1. Definition, tangent spaces, and efficiency Let \(W\) denote the observed data, \(\theta\) the parameter of interest, and \(\eta\) or \(g\) a nuisance parameter. First-order orthogonality requires a moment \(\psi(W;\theta,\eta)\) to satisfy \[ E[\psi(W;\theta_0,\eta_0)] = 0, \qquad \partial_\eta E[\psi(W;\theta_0,\eta)]\big|_{\eta=\eta_0}=0. \] In tangent-space language, this means that \(\psi(\cdot;\theta_0,\eta_0)\) lies in the orthogonal complement of the nuisance tangent space \(T_\eta\), equivalently \(E[\psi s_\eta]=0\) for all nuisance scores \(s_\eta\). This is the basic Neyman-orthogonality condition used in semiparametric efficiency theory and in double/debiased ML [2303.11418] [1711.00342]. The efficiency side is governed by the efficient score \[ S_\theta(Z)=s_\theta(Z)-\Pi_{T_\eta}s_\theta(Z), \] and the efficient Fisher information \[ I_\theta^*=E[S_\theta(Z)S_\theta(Z)']. \] When regular identification holds, the efficient influence function is obtained from the efficient score, and its variance gives the semiparametric efficiency bound. Orthogonal moments are informative about the target only when the efficient Fisher information is non-zero, though it may be singular in multivariate problems; full rank is required for regular identification, but not for existence of nontrivial orthogonal moments [2303.11418]. A closely related representation appears in nonlinear single-index conditional moment restriction models. There the naive gradient score is \[ S_{\mathrm{naive}}(W) = m\big(W,\Lambda(Z,g_0(Z))'\theta_0,g_0(Z)\big)\cdot \Lambda\big(Z,g_0(Z)\big), \] and the orthogonalized score is the projection of this object onto the orthocomplement of the nuisance tangent space. For low-dimensional targets, the efficient influence function typically takes the form \[ \varphi(W)=\Gamma^{-1}S_{\mathrm{orth}}(W), \] with \(\Gamma\) an information matrix built from the derivative \(\partial_t m\) and the index loading \(\Lambda\) [1806.04823]. ## 2. Orthogonalization in nonlinear semiparametric single-index models A central construction appears in nonlinear semiparametric single-index conditional moment restriction models of the form \[ E\big[m\big(W,\Lambda(Z,\gamma)'\theta_0,\gamma\big)\mid \gamma=g_0(Z),Z=z\big]=0,\quad \forall z, \] where the target \(\theta\in\mathbb{R}^p\) is sparse and enters the moment only through the single index \(t=\Lambda(Z,\gamma)'\theta\). The corresponding population gradient satisfies \[ \nabla_\theta Q(\theta_0,g_0) = E\Big[m\big(W,\Lambda(Z,g_0(Z))'\theta_0,g_0(Z)\big)\Lambda\big(Z,g_0(Z)\big)\Big] =0. \] The key objective is to adjust the moment so that the future loss gradient is insensitive to first-stage regularization bias while preserving this single-index structure [1806.04823]. The orthogonalization recipe starts from a preliminary moment \(m_{\mathrm{pre}}\) and a nuisance \(p_0\) identified by a conditional exogeneity restriction \[ E\big[R(W,p_0(Z))\mid Z=z\big]=0. \] Defining \[ h_0(z):=E\Big[\partial_{\gamma_1} m_{\mathrm{pre}}\mid \gamma_1=p_0(Z),\, t=\Lambda(Z,\gamma_1)'\theta_0,\, Z=z\Big], \] and \[ I_0(z):=E\Big[\partial_{\gamma_1}R(W,\gamma_1)\mid \gamma_1=p_0(Z),\, Z=z\Big], \] the orthogonalized moment is \[ m(w,t,\gamma) = m_{\mathrm{pre}}(w,t,\gamma_1)-\gamma_2\gamma_3^{-1}R(w,\gamma_1), \qquad \gamma=(\gamma_1,\gamma_2,\gamma_3), \] with nuisance output \(g_0(z)=\{p_0(z),h_0(z),I_0(z)\}\). Because the correction term does not depend on \(t\), the single-index structure is preserved. The associated loss is obtained by solving \[ \frac{\partial}{\partial t}\ell(w,t,\gamma)=m(w,t,\gamma) \] and setting \[ Q(\theta,g)=E\Big[\ell\big(W,\Lambda(Z,g(Z))'\theta,g(Z)\big)\Big]. \] Regularized estimation then uses \[ \widehat{\theta}\in \arg\min_{\theta\in\mathbb{R}^p} \Big\{ \widehat{Q}(\theta,\widehat{g})+\lambda\|\theta\|_1 \Big\}. \] Under monotonicity, identification, smoothness, and first-stage rate conditions, the estimator satisfies \[ \|\widehat{\theta}-\theta_0\|_2=O_p(\sqrt{k}\lambda), \qquad \|\widehat{\theta}-\theta_0\|_1=O_p(k\lambda), \] and under orthogonality it achieves the oracle rate when \(g_n^2=o((\log p/n)^{1/2})\) and \(\lambda\) is chosen at the moderate level \(\lambda_{\mathrm{mod}}=O(g_n^2+\sqrt{\log p/n})\) [1806.04823]. The logistic treatment-effect example makes the construction explicit. With monotone link \(G\), \[ m(w,t,\gamma)=\frac{y-G(t+\gamma_2)}{\gamma_3}, \qquad t=(d-\gamma_1)(1,x)'\theta, \] and \[ \gamma_3 = V_0(d,x) = G\!\big((d-p_0(x))(1,x)'\theta_0+q_0(x)\big) \big(1-G\!\big((d-p_0(x))(1,x)'\theta_0+q_0(x)\big)\big). \] This weighting removes first-order sensitivity to regularization bias in \(q\). The same framework also covers CCP-based models in static games of incomplete information, missing-data problems, and quantile variants [1806.04823]. ## 3. Efficient orthogonal Z-moments and the M–Z efficiency gap For scalar functionals, loss minimization and moment identification are linked by differentiation and integration. In multivariate problems this one-to-one relation fails because a vector identification function \(\psi(y,\theta)\) can be a gradient of a loss only if its Jacobian is symmetric, equivalently if the associated vector field is curl-free. This failure creates an efficiency gap: the most efficient Z-estimator can outperform the most efficient M-estimator because efficient identification functions need not have antiderivative losses [2010.14146]. In a semiparametric conditional model with strict identification function \(\phi\), the general efficient Z-estimator uses moments of the form \[ \psi_t(W_t,\theta) = A_t(X_t,\theta)\,\phi(Y_t,m(X_t,\theta)), \] with optimal instrument \[ A_{t,\star}(X_t,\theta_0) = C\cdot D_t(X_t,\theta_0)'S_t(X_t,\theta_0)^{-1}, \] where \[ S_t(X_t,\theta)=E_t[\phi(Y_t,m(X_t,\theta))\phi(Y_t,m(X_t,\theta))'], \] and \[ D_t(X_t,\theta)=\partial_\theta E_t[\phi(Y_t,m(X_t,\theta))]'. \] The efficient score-like object is \[ g_t(W_t;\theta_0) = D_t(X_t,\theta_0)'S_t(X_t,\theta_0)^{-1}\phi(Y_t,m(X_t,\theta_0)), \] and the efficient influence function is \[ \phi_t(W_t)=A_T\,g_t(W_t;\theta_0), \] where \(A_T\) is the efficiency bound matrix. These moments are Neyman-orthogonal: the nuisance derivative of their expectation vanishes at the truth [2010.14146]. The paper establishes this structure for multiple quantiles and for the \((\mathrm{VaR},\mathrm{ES})\) pair. For two quantiles, the strict identification function is \[ \phi(y;q_1,q_2)= \big(1\{y\le q_1\}-a,\;1\{y\le q_2\}-\beta\big)'. \] For \((\mathrm{VaR}_a,\mathrm{ES}_a)\), the strict identification function is \[ \phi(y;v,e) = \Big( 1\{y\le v\}-a,\; e-v+\frac{1}{a}(v-y)1\{y\le v\} \Big)'. \] In both cases the efficient instrument uses off-diagonal interactions through \(S_t^{-1}\) and \(D_t\), whereas the class of strictly consistent joint losses is more restrictive. Hence the efficient Z-estimator often attains a smaller asymptotic covariance than any M-estimator. By contrast, for \((E[Y],\mathrm{Var}[Y])\), the loss class is large enough that the M-estimator can attain the Z-efficiency bound [2010.14146]. This distinction is structural rather than algorithmic. It arises because the efficient orthogonal moment is determined by the geometry of the identification function and the nuisance tangent space, whereas M-estimation is constrained by the existence of a potential function. ## 4. Existence, informativeness, and approximation of efficient orthogonal moments A general existence theory is given by Restricted Local Non-surjectivity (RLN). Let \(T_\eta\) be the restricted nuisance tangent space and \(G_0\) the Cramér class. RLN states that there exists a nonzero \(g\in T_\eta^\perp\cap G_0\). This condition is necessary and sufficient for the existence of orthogonal or locally robust moments. It does not require identification of the parameter of interest or of the nuisance parameter. Informativeness is a separate issue: orthogonal moments are informative only if the efficient Fisher information is non-zero, though it may be singular [2303.11418]. For general smooth functionals \(v(\lambda_0)\), the same paper characterizes orthogonal moments through operator null spaces. In conditional moment models \[ E[p_j(Y,X,\theta,\eta)\mid W_j]=0,\qquad j=1,\dots,J, \] orthogonal moments take the form \[ \psi(Z;\theta_0,\eta_0)=\sum_{j=1}^J p_j(Z;\theta_0,\eta_0)\,\phi_j(W_j), \] with \(\phi\) in the null space of the adjoint operator \(M_v^*\). This yields orthogonal-relevant instruments as projections of the score of interest onto that null space. The fully saturated 2SLS score \[ \psi(Z) = (Y_1-\theta_0Y_2-\eta_0(X)) \cdot (E[Y_2\mid W]-E[Y_2\mid X]) \] is a canonical example [2303.11418]. A complementary efficiency theory applies to seemingly unrelated conditional moment restrictions with different conditioning variables, \[ E[g_j(Z,\theta_0)\mid X^{(j)}]=0,\qquad j=1,\dots,J. \] The semiparametric efficiency bound in this SUR-CMR setting is characterized as the limit of explicit efficiency bounds for a decreasing sequence of unconditional moment models. With a dense countable instrument system \(\{w_s\}\), the \(k\)-th unconditional approximation uses \[ g^{(k)}(Z,\theta)=w^{(k)}(X)\,g(Z,\theta), \] and its information matrix is \[ I^{(k)} = E[\partial_\theta g^{(k)}(Z,\theta_0)]' \,\mathrm{Var}(g^{(k)}(Z,\theta_0))^{-1}\, E[\partial_\theta g^{(k)}(Z,\theta_0)]. \] The limit of \((I^{(k)})^{-1}\) equals the SUR-CMR efficiency bound. When the efficient score is not explicit, an iterative backfitting procedure approximates the projection onto the orthocomplement of the nuisance tangent space and converges in \(L^2\) [1111.6428]. A variational route to the same target appears in conditional moment models \[ E[\rho(X;\theta_0)\mid Z]=0. \] The variational method of moments defines a minimax criterion \[ \hat\theta_n^{\mathrm{VMM}} = \arg\min_{\theta\in\Theta} \sup_{f\in\mathcal F_n} E_n[f(Z)'\rho(X;\theta)] -\frac{1}{4}E_n[(f(Z)'\rho(X;\tilde\theta_n))^2] -R_n(f), \] which recovers optimally weighted GMM when \(\mathcal F_n\) is restricted to a finite instrument span. The variational maximizer yields the efficient instrument \[ F^*(z) = V(z;\theta_0)^{-1}E[D(X;\theta_0)\mid Z=z], \] and the associated score \(F^*(Z)'\rho(X;\theta)\) is orthogonal and semiparametrically efficient when the prior estimate is consistent, \(\tilde\theta=\theta_0\) [2012.09422]. ## 5. Higher-order orthogonality and robustness beyond first order First-order orthogonality eliminates linear sensitivity to nuisance error. Higher-order orthogonality removes lower-order terms more aggressively. In general Z-estimation, \(k\)-th order orthogonality requires vanishing conditional derivatives up to order \(k\) with respect to nuisance components. Under standard regularity conditions, this relaxes sufficient nuisance-rate requirements from \(n^{-1/4}\) at first order to \(n^{-1/(2k+2)}\) at order \(k\) [1711.00342]. In partially linear regression, \[ Y=D\theta_0+g_0(X)+\varepsilon, \qquad D=m_0(X)+V, \] the standard first-order orthogonal score is \[ \psi_1(W;\theta,q,m) = (Y-q(X)-\theta(D-m(X)))(D-m(X)). \] At the truth, \(\psi_1=\varepsilon V\), the derivative with respect to \(\theta\) is \(-E[V^2]\), and the asymptotic variance is \[ \frac{E[\varepsilon^2V^2]}{(E[V^2])^2}. \] This score equals the efficient influence function scaled by \(E[V^2]\), so the cross-fitted first-order DML estimator is semiparametrically efficient [1711.00342]. A second-order score takes the form \[ \psi_2(W;\theta,q,m,\mu_{r-1},\mu_r) = (Y-q(X)-\theta(D-m(X)))\,H_r(V,X), \] with \[ H_r(V,X)=V^r-\mu_r(X)-rV\mu_{r-1}(X). \] Its nondegeneracy condition holds if and only if the conditional law of \(V\mid X\) is not almost surely Gaussian. This is the Gaussian barrier proved using Stein’s lemma. Second-order orthogonality improves robustness to slower nuisance rates, but the resulting score is generally not the efficient influence function, and its gain is rate robustness rather than semiparametric efficiency [1711.00342]. A separate higher-order construction for finite-dimensional moment-condition models builds \(q\)-th order orthogonal moments from nuisance-identifying moments \(g(W,\theta,\eta)\), target moments \(m(W,\theta,\eta)\), and a left inverse \(\Lambda\) of the nuisance Jacobian. In the affine case, \[ \psi^{(1)}(W_1,W_2;\theta,\eta,\lambda) = m(W_1,\theta,\eta) - \partial_\eta m(W_1,\theta,\eta)'\Lambda(\lambda)g(W_2,\theta,\eta), \] and an explicit \(q=2\) formula adds quadratic correction terms. In the nonlinear case, the construction is indexed by rooted trees, with closed-form coefficients ensuring cancellation of all mixed derivatives in \((\eta,\lambda)\) up to order \(q\). The added nuisance dimension is independent of \(q\) and can be reduced to a scalar through determinant transformations. The resulting bias is of order \[ O(\|\hat\eta-\eta_0\|^{q+1}+\|\hat\lambda-\lambda_0\|^{q+1}), \] and, with cross-fitting and optimal GMM weighting, the limiting variance formula remains the standard GMM one [2605.10842]. ## 6. Applications, implementation, and limitations Empirical implementations show orthogonal moments functioning as a practical interface between semiparametric efficiency theory and modern first-stage estimation. In the Connecticut Jobs First application, the target is the short-term heterogeneous impact of welfare reform on women’s welfare participation under a partially linear logistic single-index model. Nuisance components are estimated by probability random forest, random forest, and logistic regression; the orthogonal loss uses weight \[ \widehat V(d,x)=G'((d-\widehat p(x))(1,x)'\check\theta+\widehat q(x)), \] with penalty \(\lambda\approx 0.031\). The direct Lasso suggests a uniform increase in welfare participation, whereas the orthogonal Lasso detects heterogeneity: women with long prior AFDC receipt, defined as at least five months in the year pre-random assignment and measured by the covariate yrkvad, show reduced welfare participation in Jobs First [1806.04823]. Orthogonalization also applies when the nuisance is a latent fixed effect estimated from a panel first stage. In the auxiliary panel model \[ Y_{it}=X_{it}'\beta_0+\alpha_i+u_{it}, \] the cross-sectional parameter \(\mu_0\) is defined by \[ E[m(W_i,\alpha_i,\mu_0)]=0. \] The orthogonalized moment is \[ m^*(W_i,\alpha_i,Z_i,\beta_0,\mu,a) = m(W_i,\alpha_i,\mu) + a(X_i,\alpha_i,\mu)\big(Y_{iT}-X_{iT}'\beta_0-\alpha_i\big), \] with \[ a(X_i,\alpha_i,\mu_0) = E\!\left(\frac{\partial m(W_i,\alpha_i,\mu_0)}{\partial \alpha_i}\,\Big|\,X_i,\alpha_i\right). \] Under \(T\propto N^g\) with \(1/2