Efficient Orthogonal Moments in Semiparametric Models

• Semiparametrically efficient orthogonal moments are functions that are both Neyman-orthogonal to nuisance perturbations and aligned with the efficient score to attain the semiparametric variance bound.
• They are constructed by projecting moment functions onto the orthocomplement of the nuisance tangent space, preserving single-index structures in models like panel fixed effects and quantile estimation.
• This methodology underlies advanced techniques such as double/debiased machine learning, efficient Z-estimation, and higher-order orthogonality to enhance robustness and reduce estimation bias.

Semiparametrically efficient orthogonal moments are moment functions, scores, or estimating equations for a finite-dimensional target parameter that satisfy two properties simultaneously: they are Neyman-orthogonal to nuisance perturbations, so their population expectation is first-order insensitive to errors in the nuisance, and they coincide with, or induce, the efficient score or efficient influence function, so that regular estimators based on them attain the semiparametric variance lower bound. In modern usage, the concept spans conditional moment restriction models, high-dimensional regularized M- and Z-estimation, double/debiased machine learning, multivariate elicitable functionals, and generated-regressor settings with latent effects or fixed effects (Nekipelov et al., 2018, Dimitriadis et al., 2020, Argañaraz et al., 2023).

1. Definition, tangent spaces, and efficiency

Let $W$ denote the observed data, $\theta$ the parameter of interest, and $\eta$ or $g$ a nuisance parameter. First-order orthogonality requires a moment $\psi(W;\theta,\eta)$ to satisfy

$$E[\psi(W;\theta_0,\eta_0)] = 0, \qquad \partial_\eta E[\psi(W;\theta_0,\eta)]\big|_{\eta=\eta_0}=0.$$

In tangent-space language, this means that $\psi(\cdot;\theta_0,\eta_0)$ lies in the orthogonal complement of the nuisance tangent space $T_\eta$, equivalently $E[\psi s_\eta]=0$ for all nuisance scores $s_\eta$. This is the basic Neyman-orthogonality condition used in semiparametric efficiency theory and in double/debiased ML (Argañaraz et al., 2023, Mackey et al., 2017).

The efficiency side is governed by the efficient score

$\theta$0

and the efficient Fisher information

$\theta$1

When regular identification holds, the efficient influence function is obtained from the efficient score, and its variance gives the semiparametric efficiency bound. Orthogonal moments are informative about the target only when the efficient Fisher information is non-zero, though it may be singular in multivariate problems; full rank is required for regular identification, but not for existence of nontrivial orthogonal moments (Argañaraz et al., 2023).

A closely related representation appears in nonlinear single-index conditional moment restriction models. There the naive gradient score is

$\theta$2

and the orthogonalized score is the projection of this object onto the orthocomplement of the nuisance tangent space. For low-dimensional targets, the efficient influence function typically takes the form

$\theta$3

with $\theta$4 an information matrix built from the derivative $\theta$5 and the index loading $\theta$6 (Nekipelov et al., 2018).

2. Orthogonalization in nonlinear semiparametric single-index models

A central construction appears in nonlinear semiparametric single-index conditional moment restriction models of the form

$\theta$7

where the target $\theta$8 is sparse and enters the moment only through the single index $\theta$9. The corresponding population gradient satisfies

$\eta$0

The key objective is to adjust the moment so that the future loss gradient is insensitive to first-stage regularization bias while preserving this single-index structure (Nekipelov et al., 2018).

The orthogonalization recipe starts from a preliminary moment $\eta$1 and a nuisance $\eta$2 identified by a conditional exogeneity restriction

$\eta$3

Defining

$\eta$4

and

$\eta$5

the orthogonalized moment is

$\eta$6

with nuisance output $\eta$7. Because the correction term does not depend on $\eta$8, the single-index structure is preserved. The associated loss is obtained by solving

$\eta$9

and setting

$g$0

Regularized estimation then uses

$g$1

Under monotonicity, identification, smoothness, and first-stage rate conditions, the estimator satisfies

$g$2

and under orthogonality it achieves the oracle rate when $g$3 and $g$4 is chosen at the moderate level $g$5 (Nekipelov et al., 2018).

The logistic treatment-effect example makes the construction explicit. With monotone link $g$6,

$g$7

and

$g$8

This weighting removes first-order sensitivity to regularization bias in $g$9. The same framework also covers CCP-based models in static games of incomplete information, missing-data problems, and quantile variants (Nekipelov et al., 2018).

3. Efficient orthogonal Z-moments and the M–Z efficiency gap

For scalar functionals, loss minimization and moment identification are linked by differentiation and integration. In multivariate problems this one-to-one relation fails because a vector identification function $\psi(W;\theta,\eta)$0 can be a gradient of a loss only if its Jacobian is symmetric, equivalently if the associated vector field is curl-free. This failure creates an efficiency gap: the most efficient Z-estimator can outperform the most efficient M-estimator because efficient identification functions need not have antiderivative losses (Dimitriadis et al., 2020).

In a semiparametric conditional model with strict identification function $\psi(W;\theta,\eta)$1, the general efficient Z-estimator uses moments of the form

$\psi(W;\theta,\eta)$2

with optimal instrument

$\psi(W;\theta,\eta)$3

where

$\psi(W;\theta,\eta)$4

and

$\psi(W;\theta,\eta)$5

The efficient score-like object is

$\psi(W;\theta,\eta)$6

and the efficient influence function is

$\psi(W;\theta,\eta)$7

where $\psi(W;\theta,\eta)$8 is the efficiency bound matrix. These moments are Neyman-orthogonal: the nuisance derivative of their expectation vanishes at the truth (Dimitriadis et al., 2020).

The paper establishes this structure for multiple quantiles and for the $\psi(W;\theta,\eta)$9 pair. For two quantiles, the strict identification function is

$$E[\psi(W;\theta_0,\eta_0)] = 0, \qquad \partial_\eta E[\psi(W;\theta_0,\eta)]\big|_{\eta=\eta_0}=0.$$0

For $$E[\psi(W;\theta_0,\eta_0)] = 0, \qquad \partial_\eta E[\psi(W;\theta_0,\eta)]\big|_{\eta=\eta_0}=0.$$1, the strict identification function is

$$E[\psi(W;\theta_0,\eta_0)] = 0, \qquad \partial_\eta E[\psi(W;\theta_0,\eta)]\big|_{\eta=\eta_0}=0.$$2

In both cases the efficient instrument uses off-diagonal interactions through $$E[\psi(W;\theta_0,\eta_0)] = 0, \qquad \partial_\eta E[\psi(W;\theta_0,\eta)]\big|_{\eta=\eta_0}=0.$$3 and $$E[\psi(W;\theta_0,\eta_0)] = 0, \qquad \partial_\eta E[\psi(W;\theta_0,\eta)]\big|_{\eta=\eta_0}=0.$$4, whereas the class of strictly consistent joint losses is more restrictive. Hence the efficient Z-estimator often attains a smaller asymptotic covariance than any M-estimator. By contrast, for $$E[\psi(W;\theta_0,\eta_0)] = 0, \qquad \partial_\eta E[\psi(W;\theta_0,\eta)]\big|_{\eta=\eta_0}=0.$$5, the loss class is large enough that the M-estimator can attain the Z-efficiency bound (Dimitriadis et al., 2020).

This distinction is structural rather than algorithmic. It arises because the efficient orthogonal moment is determined by the geometry of the identification function and the nuisance tangent space, whereas M-estimation is constrained by the existence of a potential function.

4. Existence, informativeness, and approximation of efficient orthogonal moments

A general existence theory is given by Restricted Local Non-surjectivity (RLN). Let $$E[\psi(W;\theta_0,\eta_0)] = 0, \qquad \partial_\eta E[\psi(W;\theta_0,\eta)]\big|_{\eta=\eta_0}=0.$$6 be the restricted nuisance tangent space and $$E[\psi(W;\theta_0,\eta_0)] = 0, \qquad \partial_\eta E[\psi(W;\theta_0,\eta)]\big|_{\eta=\eta_0}=0.$$7 the Cramér class. RLN states that there exists a nonzero $$E[\psi(W;\theta_0,\eta_0)] = 0, \qquad \partial_\eta E[\psi(W;\theta_0,\eta)]\big|_{\eta=\eta_0}=0.$$8. This condition is necessary and sufficient for the existence of orthogonal or locally robust moments. It does not require identification of the parameter of interest or of the nuisance parameter. Informativeness is a separate issue: orthogonal moments are informative only if the efficient Fisher information is non-zero, though it may be singular (Argañaraz et al., 2023).

For general smooth functionals $$E[\psi(W;\theta_0,\eta_0)] = 0, \qquad \partial_\eta E[\psi(W;\theta_0,\eta)]\big|_{\eta=\eta_0}=0.$$9, the same paper characterizes orthogonal moments through operator null spaces. In conditional moment models

$\psi(\cdot;\theta_0,\eta_0)$0

orthogonal moments take the form

$\psi(\cdot;\theta_0,\eta_0)$1

with $\psi(\cdot;\theta_0,\eta_0)$2 in the null space of the adjoint operator $\psi(\cdot;\theta_0,\eta_0)$3. This yields orthogonal-relevant instruments as projections of the score of interest onto that null space. The fully saturated 2SLS score

$\psi(\cdot;\theta_0,\eta_0)$4

is a canonical example (Argañaraz et al., 2023).

A complementary efficiency theory applies to seemingly unrelated conditional moment restrictions with different conditioning variables,

$\psi(\cdot;\theta_0,\eta_0)$5

The semiparametric efficiency bound in this SUR-CMR setting is characterized as the limit of explicit efficiency bounds for a decreasing sequence of unconditional moment models. With a dense countable instrument system $\psi(\cdot;\theta_0,\eta_0)$6, the $\psi(\cdot;\theta_0,\eta_0)$7-th unconditional approximation uses

$\psi(\cdot;\theta_0,\eta_0)$8

and its information matrix is

$\psi(\cdot;\theta_0,\eta_0)$9

The limit of $T_\eta$0 equals the SUR-CMR efficiency bound. When the efficient score is not explicit, an iterative backfitting procedure approximates the projection onto the orthocomplement of the nuisance tangent space and converges in $T_\eta$1 (Hristache et al., 2011).

A variational route to the same target appears in conditional moment models

$T_\eta$2

The variational method of moments defines a minimax criterion

$T_\eta$3

which recovers optimally weighted GMM when $T_\eta$4 is restricted to a finite instrument span. The variational maximizer yields the efficient instrument

$T_\eta$5

and the associated score $T_\eta$6 is orthogonal and semiparametrically efficient when the prior estimate is consistent, $T_\eta$7 (Bennett et al., 2020).

5. Higher-order orthogonality and robustness beyond first order

First-order orthogonality eliminates linear sensitivity to nuisance error. Higher-order orthogonality removes lower-order terms more aggressively. In general Z-estimation, $T_\eta$8-th order orthogonality requires vanishing conditional derivatives up to order $T_\eta$9 with respect to nuisance components. Under standard regularity conditions, this relaxes sufficient nuisance-rate requirements from $E[\psi s_\eta]=0$0 at first order to $E[\psi s_\eta]=0$1 at order $E[\psi s_\eta]=0$2 (Mackey et al., 2017).

In partially linear regression,

$E[\psi s_\eta]=0$3

the standard first-order orthogonal score is

$E[\psi s_\eta]=0$4

At the truth, $E[\psi s_\eta]=0$5, the derivative with respect to $E[\psi s_\eta]=0$6 is $E[\psi s_\eta]=0$7, and the asymptotic variance is

$E[\psi s_\eta]=0$8

This score equals the efficient influence function scaled by $E[\psi s_\eta]=0$9, so the cross-fitted first-order DML estimator is semiparametrically efficient (Mackey et al., 2017).

A second-order score takes the form

$s_\eta$0

with

$s_\eta$1

Its nondegeneracy condition holds if and only if the conditional law of $s_\eta$2 is not almost surely Gaussian. This is the Gaussian barrier proved using Stein’s lemma. Second-order orthogonality improves robustness to slower nuisance rates, but the resulting score is generally not the efficient influence function, and its gain is rate robustness rather than semiparametric efficiency (Mackey et al., 2017).

A separate higher-order construction for finite-dimensional moment-condition models builds $s_\eta$3-th order orthogonal moments from nuisance-identifying moments $s_\eta$4, target moments $s_\eta$5, and a left inverse $s_\eta$6 of the nuisance Jacobian. In the affine case,

$s_\eta$7

and an explicit $s_\eta$8 formula adds quadratic correction terms. In the nonlinear case, the construction is indexed by rooted trees, with closed-form coefficients ensuring cancellation of all mixed derivatives in $s_\eta$9 up to order $\theta$00. The added nuisance dimension is independent of $\theta$01 and can be reduced to a scalar through determinant transformations. The resulting bias is of order

$\theta$02

and, with cross-fitting and optimal GMM weighting, the limiting variance formula remains the standard GMM one (Bonhomme et al., 11 May 2026).

6. Applications, implementation, and limitations

Empirical implementations show orthogonal moments functioning as a practical interface between semiparametric efficiency theory and modern first-stage estimation. In the Connecticut Jobs First application, the target is the short-term heterogeneous impact of welfare reform on women’s welfare participation under a partially linear logistic single-index model. Nuisance components are estimated by probability random forest, random forest, and logistic regression; the orthogonal loss uses weight

$\theta$03

with penalty $\theta$04. The direct Lasso suggests a uniform increase in welfare participation, whereas the orthogonal Lasso detects heterogeneity: women with long prior AFDC receipt, defined as at least five months in the year pre-random assignment and measured by the covariate yrkvad, show reduced welfare participation in Jobs First (Nekipelov et al., 2018).

Orthogonalization also applies when the nuisance is a latent fixed effect estimated from a panel first stage. In the auxiliary panel model

$\theta$05

the cross-sectional parameter $\theta$06 is defined by

$\theta$07

The orthogonalized moment is

$\theta$08

with

$\theta$09

Under $\theta$10 with $\theta$11, cross-fitting adapted to panel structure, and regularity conditions, the resulting estimator is $\theta$12-normal with the same first-order limit as if $\theta$13 and $\theta$14 were known. The central limit theorem does not rely on exogeneity between panel residuals and cross-sectional moment functions (Huang, 9 Feb 2026).

Implementation is consistently organized around cross-fitting or sample splitting. In the single-index regularized setting, nuisance functions are trained on auxiliary folds and the orthogonal loss is evaluated on held-out folds; admissible penalties differ sharply between orthogonal and non-orthogonal scores: $\theta$15 In multivariate Z-estimation, densities at model-implied quantiles and truncated tail variances enter $\theta$16 and $\theta$17 and are estimated with cross-fitting. In VMM, kernel or neural critics implement the optimal instrument variationally, and inference uses plug-in estimates of the efficient information matrix $\theta$18 (Nekipelov et al., 2018, Dimitriadis et al., 2020, Bennett et al., 2020).

Several limitations recur across the literature. In multivariate functional problems, not every efficient identification function has an antiderivative loss, so efficient orthogonal moments may exist only on the Z-estimation side. In nonlinear single-index models, preserving the index structure can fail if the correction term depends on the index $\theta$19, which is why the proposed construction insists on $\theta$20-independent adjustments. High-dimensional inference is typically available only for low-dimensional components or linear functionals through debiasing. Higher-order orthogonality relaxes nuisance-rate conditions, but it increases the number of independent copies in the moment kernel and can raise computational variance. These limitations are structural features of the geometry of nuisance tangent spaces, identification maps, and efficient instruments rather than artifacts of a particular estimator (Dimitriadis et al., 2020, Nekipelov et al., 2018, Bonhomme et al., 11 May 2026).

Orthogonal moments are therefore best viewed as a unifying semiparametric device: they encode nuisance insensitivity through tangent-space orthogonality, achieve efficiency when matched to the efficient score or influence function, and remain constructible in diverse settings ranging from single-index welfare models and panel fixed effects to vector quantiles, VaR–ES, and general conditional moment restrictions.