Neyman-Orthogonal Moments

• Neyman-orthogonal moments are defined as moment functions that remain insensitive to small perturbations in nuisance parameters, ensuring first-order bias cancellation.
• They are central to semiparametric inference and double machine learning, linking influence functions, efficiency, and cross-fitting to enable robust estimation of low-dimensional targets.
• Higher-order extensions of orthogonality further enhance robustness limits and have practical applications in ATE estimation, Bayesian inference, and high-dimensional survival analysis.

Neyman-orthogonal moments are moment or score functions constructed so that small perturbations of nuisance parameters do not change the first-order expectation of the estimating equation at the truth. In semiparametric inference and debiased or double machine learning, this property is used to suppress first-stage bias from high-dimensional, nonparametric, or machine-learning nuisance estimators while retaining $\sqrt{n}$-scale inference for low-dimensional targets. The modern literature treats orthogonal moments as a unifying device linking semiparametric efficiency, influence-function constructions, Riesz representers, balancing methods, higher-order robustness, and extensions to Bayesian and survival settings (Kato, 7 May 2026, Argañaraz et al., 2023, Mackey et al., 2017).

1. Definition and canonical construction

A standard semiparametric setup posits observable data $Z$, a finite-dimensional parameter of interest $\theta$, an infinite-dimensional nuisance $\eta$, and a moment function

$g(Z;\theta,\eta)\in\mathbb R^k$

such that

$\E_{P_{\theta_0,\eta_0}}\bigl[g(Z;\theta_0,\eta_0)\bigr]=0.$

Neyman orthogonality is the requirement that, for every nuisance direction $h$, the Gateaux derivative of the population moment with respect to nuisance perturbations vanishes at the truth: $\left.\frac{d}{dt}\right|_{t=0}\; \E_{P_{\theta_0,\eta_0+t\,h}} \bigl[g(Z;\theta_0,\eta_0+t\,h)\bigr]=0.$ Equivalently, when differentiation under the integral sign is valid and $S_\eta(Z)[h]$ denotes the nuisance score, orthogonality is the condition

$\E\bigl[g(Z;\theta_0,\eta_0)\,S_\eta(Z)[h]\bigr]=0 \quad \forall\,S_\eta(\cdot)[h]\in T_{\eta_0},$

with $Z$0 the nuisance tangent space (Argañaraz et al., 2023).

A particularly transparent construction arises when the target is a linear functional of a regression function. Let $Z$1, let $Z$2, and suppose

$Z$3

with $Z$4 linear in $Z$5. By the Riesz representation theorem in $Z$6 there exists $Z$7 such that

$Z$8

for every $Z$9. The associated orthogonal score is

$\theta$0

By construction,

$\theta$1

and $\theta$2 solves $\theta$3 (Kato, 7 May 2026).

This formulation places orthogonal moments at the intersection of influence-function theory and linear functional estimation. The nuisance pair $\theta$4 is not ancillary to the score; it is the mechanism through which the score neutralizes first-order nuisance perturbations.

2. Gateaux orthogonality and first-stage insensitivity

The defining operational feature of a Neyman-orthogonal moment is first-order insensitivity to nuisance estimation error. In the Riesz-based construction above, for any perturbation direction $\theta$5,

$\theta$6

The cancellation follows from two facts recorded in the literature: first, $\theta$7 is mean-zero under the model; second, the pathwise derivative of the linear functional is represented by the Riesz representer, so the derivative contributions in $\theta$8 offset each other exactly (Kato, 7 May 2026).

This property underlies the usual plug-in equation

$\theta$9

If $\eta$0 is orthogonal, a Taylor expansion in $\eta$1 shows that

$\eta$2

is of second order in $\eta$3, so even a relatively slow $\eta$4-rate estimate of $\eta$5 perturbs the moment by $\eta$6. This is the standard first-order robustness statement behind double or debiased machine learning (Sabbagh et al., 23 Feb 2026). In the formulation of higher-order orthogonality, first-order orthogonality corresponds to the case $\eta$7, and the familiar $\eta$8 requirement appears as the special case of the general rate $\eta$9 (Mackey et al., 2017).

Cross-fitting is repeatedly used to operationalize this insensitivity. In the high-dimensional and higher-order developments, cross-fitting separates nuisance training from score evaluation so that the leading empirical process is driven by the orthogonal score at the truth rather than by overfitting artifacts. In the terminology of the higher-order theory, without cross-fitting one pays a first-stage bias penalty that can destroy the $g(Z;\theta,\eta)\in\mathbb R^k$0 rate (Mackey et al., 2017).

3. ATE scores, Riesz representers, and balancing

A canonical example is the average treatment effect under binary treatment. In the notation $g(Z;\theta,\eta)\in\mathbb R^k$1 with $g(Z;\theta,\eta)\in\mathbb R^k$2, define

$g(Z;\theta,\eta)\in\mathbb R^k$3

The ATE

$g(Z;\theta,\eta)\in\mathbb R^k$4

solves

$g(Z;\theta,\eta)\in\mathbb R^k$5

where the efficient influence function is

$g(Z;\theta,\eta)\in\mathbb R^k$6

At $g(Z;\theta,\eta)\in\mathbb R^k$7, the derivative in the directions $g(Z;\theta,\eta)\in\mathbb R^k$8 is zero, so the AIPW score is Neyman-orthogonal (Sabbagh et al., 23 Feb 2026).

The recent balancing literature reframes this construction in terms of the error term that enters the orthogonal score. In the heterogeneous ATE problem with regressor $g(Z;\theta,\eta)\in\mathbb R^k$9 and outcome regression $\E_{P_{\theta_0,\eta_0}}\bigl[g(Z;\theta_0,\eta_0)\bigr]=0.$0, the deterministic part of the error of the sample-moment equation can be written in terms of the regression error $\E_{P_{\theta_0,\eta_0}}\bigl[g(Z;\theta_0,\eta_0)\bigr]=0.$1 as

$\E_{P_{\theta_0,\eta_0}}\bigl[g(Z;\theta_0,\eta_0)\bigr]=0.$2

When $\E_{P_{\theta_0,\eta_0}}\bigl[g(Z;\theta_0,\eta_0)\bigr]=0.$3 lies in the span of a finite set of covariate-only functions $\E_{P_{\theta_0,\eta_0}}\bigl[g(Z;\theta_0,\eta_0)\bigr]=0.$4, covariate-balancing constraints force the deterministic bias to vanish on that subspace. This is exactly how entropy- or kernel-based covariate balancing methods work when the relevant regression error depends only on $\E_{P_{\theta_0,\eta_0}}\bigl[g(Z;\theta_0,\eta_0)\bigr]=0.$5 (Kato, 7 May 2026).

The position advanced in recent work is more specific than a general endorsement of balancing. The argument is that, in debiased machine learning, balancing functions should be derived from the Neyman orthogonal score, not chosen only as functions of covariates. For ATE estimation under treatment effect heterogeneity, the score error generally contains treatment-specific components because the outcome regression is a function of the full regressor $\E_{P_{\theta_0,\eta_0}}\bigl[g(Z;\theta_0,\eta_0)\bigr]=0.$6. In that case, balancing common functions of $\E_{P_{\theta_0,\eta_0}}\bigl[g(Z;\theta_0,\eta_0)\bigr]=0.$7 can leave the treatment-specific component unbalanced. The proposed general principle is therefore regressor balancing, implemented by Riesz regression with basis functions of $\E_{P_{\theta_0,\eta_0}}\bigl[g(Z;\theta_0,\eta_0)\bigr]=0.$8: $\E_{P_{\theta_0,\eta_0}}\bigl[g(Z;\theta_0,\eta_0)\bigr]=0.$9 Its first-order condition implies

$h$0

so in the unregularized case $h$1 it exactly balances each $h$2. Covariate balancing is therefore not treated as invalid; it is treated as the special case appropriate when the score-relevant regression error is a function of covariates alone (Kato, 7 May 2026).

4. Existence, restricted local non-surjectivity, and information

The existence of orthogonal moments is not automatic in general semiparametric models. The central criterion introduced in recent theory is Restricted Local Non-surjectivity (RLN). Under the null $h$3, let

$h$4

be the restricted nuisance tangent space, with closure $h$5 in $h$6. The model satisfies Restricted Local Surjectivity if $h$7, and otherwise satisfies RLN. In operator language,

$h$8

is the RLN condition (Argañaraz et al., 2023).

The main existence theorem states that, under standard regularity, the model satisfies RLN if and only if there exists a nonzero orthogonal moment: $h$9 Equivalently,

$\left.\frac{d}{dt}\right|_{t=0}\; \E_{P_{\theta_0,\eta_0+t\,h}} \bigl[g(Z;\theta_0,\eta_0+t\,h)\bigr]=0.$0

An adjoint formulation gives the same criterion as

$\left.\frac{d}{dt}\right|_{t=0}\; \E_{P_{\theta_0,\eta_0+t\,h}} \bigl[g(Z;\theta_0,\eta_0+t\,h)\bigr]=0.$1

This result separates the existence question from identification of the parameter of interest and from identification of the nuisance parameter: RLN does not require either one (Argañaraz et al., 2023).

Existence, however, is not the same as informativeness. Let

$\left.\frac{d}{dt}\right|_{t=0}\; \E_{P_{\theta_0,\eta_0+t\,h}} \bigl[g(Z;\theta_0,\eta_0+t\,h)\bigr]=0.$2

be the efficient score for $\left.\frac{d}{dt}\right|_{t=0}\; \E_{P_{\theta_0,\eta_0+t\,h}} \bigl[g(Z;\theta_0,\eta_0+t\,h)\bigr]=0.$3, and let

$\left.\frac{d}{dt}\right|_{t=0}\; \E_{P_{\theta_0,\eta_0+t\,h}} \bigl[g(Z;\theta_0,\eta_0+t\,h)\bigr]=0.$4

An orthogonal moment has nontrivial local power if and only if $\left.\frac{d}{dt}\right|_{t=0}\; \E_{P_{\theta_0,\eta_0+t\,h}} \bigl[g(Z;\theta_0,\eta_0+t\,h)\bigr]=0.$5, equivalently if and only if the efficient Fisher Information matrix is non-zero, though possibly singular. In the scalar case, “nonzero” and “full-rank” coincide; in the multivariate case, a singular information matrix corresponds to semi-identification along certain directions (Argañaraz et al., 2023).

This distinction addresses a recurrent misconception. Orthogonal moments can exist in abundance, yet still fail to be informative for the parameter of interest if the efficient score vanishes. Conversely, the existence criterion explains why orthogonal moments can be found in models with unobserved heterogeneity, conditional moment restrictions with possibly different conditioning variables, fully saturated two stage least squares, heterogeneous parameters in treatment effects, sample selection models, and popular models of demand for differentiated products (Argañaraz et al., 2023).

5. Higher-order orthogonality and robustness limits

First-order Neyman orthogonality can be generalized to $\left.\frac{d}{dt}\right|_{t=0}\; \E_{P_{\theta_0,\eta_0+t\,h}} \bigl[g(Z;\theta_0,\eta_0+t\,h)\bigr]=0.$6-th order orthogonality. Let $\left.\frac{d}{dt}\right|_{t=0}\; \E_{P_{\theta_0,\eta_0+t\,h}} \bigl[g(Z;\theta_0,\eta_0+t\,h)\bigr]=0.$7 be the nuisance and let $\left.\frac{d}{dt}\right|_{t=0}\; \E_{P_{\theta_0,\eta_0+t\,h}} \bigl[g(Z;\theta_0,\eta_0+t\,h)\bigr]=0.$8 be a multi-index. A moment $\left.\frac{d}{dt}\right|_{t=0}\; \E_{P_{\theta_0,\eta_0+t\,h}} \bigl[g(Z;\theta_0,\eta_0+t\,h)\bigr]=0.$9 is $S_\eta(Z)[h]$0-orthogonal at $S_\eta(Z)[h]$1 if

$S_\eta(Z)[h]$2

A Taylor expansion of the plug-in moment around $S_\eta(Z)[h]$3 then removes all nuisance terms up to order $S_\eta(Z)[h]$4, leaving only the $S_\eta(Z)[h]$5-st order remainder. Under standard regularity and cross-fitting, if $S_\eta(Z)[h]$6 estimates $S_\eta(Z)[h]$7 at rate

$S_\eta(Z)[h]$8

then

$S_\eta(Z)[h]$9

For $\E\bigl[g(Z;\theta_0,\eta_0)\,S_\eta(Z)[h]\bigr]=0 \quad \forall\,S_\eta(\cdot)[h]\in T_{\eta_0},$0 this recovers the $\E\bigl[g(Z;\theta_0,\eta_0)\,S_\eta(Z)[h]\bigr]=0 \quad \forall\,S_\eta(\cdot)[h]\in T_{\eta_0},$1 first-order result; for $\E\bigl[g(Z;\theta_0,\eta_0)\,S_\eta(Z)[h]\bigr]=0 \quad \forall\,S_\eta(\cdot)[h]\in T_{\eta_0},$2 the required nuisance rate becomes $\E\bigl[g(Z;\theta_0,\eta_0)\,S_\eta(Z)[h]\bigr]=0 \quad \forall\,S_\eta(\cdot)[h]\in T_{\eta_0},$3 (Mackey et al., 2017).

The partially linear regression model is the principal explicit case: $\E\bigl[g(Z;\theta_0,\eta_0)\,S_\eta(Z)[h]\bigr]=0 \quad \forall\,S_\eta(\cdot)[h]\in T_{\eta_0},$4

$\E\bigl[g(Z;\theta_0,\eta_0)\,S_\eta(Z)[h]\bigr]=0 \quad \forall\,S_\eta(\cdot)[h]\in T_{\eta_0},$5

In this setting, second-order orthogonal moments can be constructed if and only if the treatment residual is not normally distributed. The characterization is

$\E\bigl[g(Z;\theta_0,\eta_0)\,S_\eta(Z)[h]\bigr]=0 \quad \forall\,S_\eta(\cdot)[h]\in T_{\eta_0},$6

if and only if the conditional distribution of $\E\bigl[g(Z;\theta_0,\eta_0)\,S_\eta(Z)[h]\bigr]=0 \quad \forall\,S_\eta(\cdot)[h]\in T_{\eta_0},$7 is Gaussian almost surely. Hence a violation such as non-zero skewness or excess kurtosis permits second-order orthogonal moments; conditional Gaussianity implies that no non-degenerate second-order orthogonal moment exists (Mackey et al., 2017).

This yields a precise limitation rather than a generic promise of arbitrary robustness. Higher-order orthogonality can improve the admissible nuisance rate, but it requires deeper moment constructions and stronger smoothness. The same source states that in high-dimensional linear nuisances, second-order orthogonality can tolerate sparsity up to $\E\bigl[g(Z;\theta_0,\eta_0)\,S_\eta(Z)[h]\bigr]=0 \quad \forall\,S_\eta(\cdot)[h]\in T_{\eta_0},$8 versus $\E\bigl[g(Z;\theta_0,\eta_0)\,S_\eta(Z)[h]\bigr]=0 \quad \forall\,S_\eta(\cdot)[h]\in T_{\eta_0},$9 for first-order, while in nonparametric settings the needed $Z$00 rate may require very strong smoothness (Mackey et al., 2017).

6. Bayesian and high-dimensional survival extensions

Neyman orthogonality has also been exported beyond the conventional frequentist DML setting. In semi-parametric Bayesian inference with a non-parametrically modelled nuisance component, the relevant result is that when the nuisance and targeted parameters satisfy a Neyman orthogonal score property, cutting feedback through a two-step procedure is a valid way of conducting Bayesian inference. Using a Dirichlet process and the Bayesian bootstrap, one obtains that the marginal posterior of the targeted parameter exhibits good frequentist properties despite not accounting for the inferential uncertainty of the nuisance parameter. For the plug-in estimator $Z$01 and the Bayesian-bootstrap draw $Z$02,

$Z$03

with the same asymptotic variance. The same work also investigates the absence of Neyman orthogonality and shows that, for a simple family of useful scores, the posterior distribution is asymptotically unchanged by nuisance estimation provided the nuisance estimator is consistent (Sabbagh et al., 23 Feb 2026).

In survival analysis, orthogonality is used to debias high-dimensional hazard-based estimators. The HSCI framework observes

$Z$04

assumes unconfoundedness and independent censoring given $Z$05, and posits a sparse high-dimensional Cox proportional hazards outcome model together with a high-dimensional logistic propensity score working model. The Neyman near-orthogonal score is

$Z$06

with

$Z$07

chosen so that

$Z$08

Implemented with cross-fitting, the framework establishes root-n asymptotic normality and consistent variance estimation under doubly robust nuisance-rate conditions: $Z$09 A Wald-type $Z$10 confidence interval is

$Z$11

The same framework extends to inference on high-dimensional survival covariate effects by one-step correction of the Lasso estimator (Fan et al., 12 Jun 2026).

Across these extensions, the recurring pattern is stable. Orthogonality is used to decouple the target parameter from first-order nuisance distortion, but the technical realization varies with the inferential regime: efficient influence functions in causal inference, RLN and tangent-space geometry in existence theory, higher-order derivatives in robustness analysis, Dirichlet-process and Bayesian-bootstrap arguments in Bayesian semiparametrics, and near-orthogonal score correction in high-dimensional Cox models. A plausible implication is that Neyman-orthogonal moments are best viewed not as a single estimator, but as a design principle for constructing inferentially stable estimating equations in semiparametric problems.