Neyman Orthogonal Moments in Semiparametric Inference

• Neyman orthogonal moments are estimating functions defined to cancel first-order bias from nonparametric nuisance parameter estimation, ensuring local robustness.
• They are constructed via projections in tangent spaces, facilitating debiased estimation and √n-consistency through rigorous semiparametric techniques.
• Applications span double machine learning and robust Bayesian inference, where these moments enable accurate parameter estimation despite high-dimensional nuisance components.

Neyman orthogonal moments are specially constructed estimating functions in semiparametric and nonparametric inference that exhibit first-order insensitivity to misspecification or estimation error in high- or infinite-dimensional nuisance parameters. The key property of these moments, termed Neyman orthogonality or local robustness, is that the influence of estimation error from arbitrary plug-in or machine learning methods for the nuisance component enters inference on the low-dimensional target parameter only at second (or higher) order, providing rigorous guarantees for debiased or robust estimation even when nuisance parameters are nonparametrically or adaptively estimated. The framework for Neyman orthogonal moments underpins key developments in modern semiparametric theory, double machine learning, and robust two-step Bayesian inference.

1. Formal Definition and Characterization

Let $O \in \mathcal{O}$ be observed data, $P_0$ its law, $\theta_0 \in \Theta \subset \mathbb{R}^p$ the low-dimensional target, and $h_0 \in \mathcal{H}$ an infinite-dimensional nuisance parameter. Neyman orthogonal moments are defined through a score or estimating function $m(O; \theta, h)$ characterizing the target by the population moment condition

$\mathbb{E}_{P_0}[m(O; \theta_0, h_0)] = 0.$

The Neyman orthogonality requirement is that the pathwise (or Gâteaux) directional derivative of the moment function with respect to $h$ at $h_0$ vanishes for all possible directions: $$\left. \frac{d}{dt} \mathbb{E}_{P_0}[m(O; \theta_0, h_0 + t(h - h_0))]\right|_{t=0} = 0 \quad \forall\, h \in \mathcal{H}.$$ In more abstract terms, for any path $(\theta_0, h_t)$ with $h_0$ fixed, the mean of the moment function is locally flat in $h$ at $h_0$, i.e., estimating $h_0$ at a slow rate does not affect the first-order behavior of the estimating equation for $\theta_0$ (Mackey et al., 2017, Sabbagh et al., 23 Feb 2026, Argañaraz et al., 2023).

2. Existence and General Construction via Tangent Spaces

A necessary and sufficient condition for the existence of nontrivial Neyman orthogonal moments is Restricted Local Non-surjectivity (RLN). For a regular semiparametric model $P_{\theta, \eta}$, the tangent space $T_0$ is the closure (in $L^2$) of scores for paths varying only the nuisance, with the orthocomplement $T_0^\perp$ containing all functions orthogonal to all nuisance scores. Nonzero elements in $T_0^\perp$ are precisely Neyman orthogonal moments. Formally, RLN is satisfied (i.e., $T_0^\perp \neq \{0\}$) whenever the nuisance tangent space is not dense in $L^2$, ensuring that the parameter of interest is not 'absorbed' by nuisance variation (Argañaraz et al., 2023).

In parametric or semiparametric models, the classical Neyman orthogonal score takes the form

$$\psi(Z;\theta, \eta) = s_\theta(Z; \theta, \eta) - A\,s_\eta(Z; \theta, \eta),$$

where $$A = \mathbb{E}[s_\theta s_\eta']\,\mathbb{E}[s_\eta s_\eta']^{-1}$$, ensuring mean insensitivity to infinitesimal changes in $\eta$. For general models, the efficient orthogonal score is the $L^2$-projection of $s_\theta$ onto $T_0^\perp$.

3. Inference Properties and the Bernstein–von Mises Theorem

The central utility of Neyman orthogonal moments lies in their first-order robustness to plug-in estimation of the nuisance parameter. For any estimator $\hat h$ with $\|\hat h - h_0\| = o_P(n^{-1/4})$, the leading bias due to the estimation of $h_0$ cancels in large samples, and the estimator for $\theta_0$ achieves $\sqrt n$-consistency and asymptotic normality. Specifically, after a Taylor expansion and under Neyman orthogonality,

$$n^{1/2}(\hat \theta_n - \theta_0) = -M_{\theta_0}^{-1} n^{-1/2} \sum_{i=1}^n m(O_i; \theta_0, h_0) + o_P(1),$$

where $$M_{\theta_0} = \mathbb{E}_{P_0}[\partial_\theta m(O;\theta_0,h_0)]$$. The induced marginal posterior (e.g., via Bayesian bootstrap with plug-in $h$) satisfies a Bernstein–von Mises theorem: $$\pi_n(\theta \mid data) \approx N\left(\hat \theta_n, \frac{1}{n}\Sigma\right),$$ with $$\Sigma = M_{\theta_0}^{-1} \mathbb{E}_{P_0}[m m^\top] M_{\theta_0}^{-T}$$. Thus, credible intervals coincide to first order with frequentist intervals even when inferential uncertainty in $h$ is ignored (Sabbagh et al., 23 Feb 2026, Mackey et al., 2017).

4. Higher-Order Orthogonality and Rate Relaxation

While first-order Neyman orthogonality ensures bias insensitivity to $O(n^{-1/4})$ estimation rates for the nuisance, higher-order orthogonality (or $k$-th order S-orthogonality) is achievable by constructing moments such that all mixed derivatives up to order $k$ vanish in expectation. Then, plug-in bias enters at $O(\|\hat \eta - \eta_0\|^{k+1})$, and consistent estimation requires only $\|\hat \eta - \eta_0\| = o(n^{-1/(2k+2)})$. In partially linear regression models, a genuine second-order orthogonal moment exists if and only if the treatment residual is non-Gaussian (proved using Stein's Lemma). Under these constructions, one can accommodate denser, more complex nuisance components or relax the required convergence rate for plug-in estimation (Mackey et al., 2017).

5. Practical Algorithms: Plug-In, Bayesian Bootstrap, and "Cutting Feedback"

Implementation often proceeds by first estimating the nuisance $h_0$ (potentially by machine learning) and then solving the empirical moment equation with $\hat h$ fixed, using frequentist, Bayesian, or bootstrap procedures. Notably, in the Bayesian framework, posterior draws can be obtained by (i) fixing $\hat h$ and (ii) solving the bootstrapped moment equation for each set of bootstrap weights, e.g., under the Bayesian bootstrap ($w_{in}\sim \mathrm{Dirichlet}(1,\dots,1)$). Neyman orthogonality ensures that "cutting feedback"—not updating $h$ in the second stage—retains valid inference for the target parameter to first order, which is critical in models where feedback between $h$ and $\theta$ is computationally burdensome or conceptually undesirable (Sabbagh et al., 23 Feb 2026).

6. Applications and Model-Specific Constructions

Neyman orthogonal moments are foundational in semiparametric causal inference, treatment effect and policy learning, double/debiased machine learning, and econometric models with high-dimensional or nonparametric nuisance structure. Concrete examples include:

• Partially linear regression, where orthogonal moments enable valid inference on treatment effects with high-dimensional confounding (Mackey et al., 2017)
• Mixture models and models with unobserved heterogeneity, where functional-differencing yields nuisance-free orthogonal moments (Argañaraz et al., 2023)
• Two-stage least squares (2SLS), sample selection models, and BLP demand systems, where explicit orthogonal moment forms can be constructed via projection methods (Argañaraz et al., 2023)
• Bayesian semiparametrics (using Dirichlet process priors), where valid marginal posteriors for parameters of interest are obtained by plug-in, under orthogonality (Sabbagh et al., 23 Feb 2026)

These methods unify a wide array of robust and debiased inference tools now standard in both econometrics and statistical machine learning.

7. Limitations and Extensions Beyond Orthogonality

If the Neyman orthogonality condition fails (i.e., if $f'(0) \neq 0$ for the pathwise mean), the resulting inference for $\theta_0$ incurs a first-order bias proportional to the estimation rate of $\hat h$, and credible/confidence intervals may not have correct frequentist coverage absent explicit bias correction. Remarkably, for some "linear" moment conditions, the asymptotic variance remains correct under mere consistency of $\hat h$, but centering is biased; thus, error control and coverage require careful adjustment (Sabbagh et al., 23 Feb 2026, Mackey et al., 2017). The existence condition RLN is very weak; in typical models it is automatically satisfied, but power and informativeness still require the efficient Fisher information for $\theta$ to be nonzero, though possibly singular (Argañaraz et al., 2023).

In summary, Neyman orthogonal moments are a cornerstone of robust semiparametric inference, yielding procedures with insensitivity to high-dimensional or nonparametric nuisance estimation, and have led to major advances across modern statistics and empirical economics (Sabbagh et al., 23 Feb 2026, Mackey et al., 2017, Argañaraz et al., 2023).