---
title: Debiased Machine Learning (DML)
url: https://www.emergentmind.com/topics/debiased-machine-learning-dml
type: topic
---

# Debiased Machine Learning (DML)

Searching arXiv for recent and foundational papers on debiased machine learning to ground the article.
Debiased Machine Learning (DML) is a framework for estimating low‑dimensional causal and structural parameters in models with high‑dimensional or complex nuisance components by constructing **orthogonal (Neyman‑orthogonal, doubly robust) score/moment functions** that are insensitive to small errors in nuisance estimation, estimating these nuisance functions using **flexible ML methods**, and applying **cross‑fitting (sample splitting)** so that the resulting estimator of the target parameter is root‑\(n\) consistent, asymptotically normal, and amenable to valid inference [1701.08687]. In the formulations emphasized across recent work, DML is also a **meta algorithm** and a **finite-sample theorem** for functionals of machine learning predictors, and a broader view treats nuisance estimation itself as a target of score design through Riesz representers, Bregman divergence minimization, covariate balancing, and TMLE‑style updating [2105.15197], [2510.23534].

## 1. Conceptual foundations

The basic setting is that one observes i.i.d. data \(W=(X,Y)\) or \(W=(Y,D,Z)\), and the parameter of interest is a scalar functional of an unknown regression function. A canonical representation is
\[
\theta_0 = E\big[m(W,\gamma_0)\big],
\]
where \(\gamma_0(x)=E[Y\mid X=x]\) and \(m\) is linear in \(\gamma\) [2510.23534], [2105.15197]. In treatment-effect settings, examples include the average treatment effect
\[
\theta_0 = E[\gamma_0(1,V,X) - \gamma_0(0,V,X)],
\]
while for local functionals one considers kernel‑weighted limits such as
\[
\theta_0^{\lim} = \lim_{h\to 0} E[\ell_h(W_j)m(W,\gamma_0)].
\]
The common difficulty is that naïvely plugging a flexible ML estimate \(\widehat\gamma\) into the identifying functional can yield poor performance because ML estimators converge slower than \(n^{-1/2}\) and the functional is not “smooth” enough [2510.23534], [2105.15197].

The central device is a **Neyman orthogonal score**. In the linear-functional framework, the score is
\[
\psi(W;\eta,\theta) = m(W,\gamma) + \alpha(X)\{Y-\gamma(X)\} - \theta,
\]
where \(\eta=(\gamma,\alpha)\) and \(\alpha\) is the **Riesz representer** [2510.23534]. At the truth,
\[
E\big[\psi(W;\eta_0,\theta_0)\big] = 0,
\]
and for any perturbation direction \(h\),
\[
\partial_{\eta} E\big[\psi(W;\eta,\theta_0)\big]\big|_{\eta=\eta_0}[h] = 0.
\]
This orthogonality means that small errors in \(\eta\) do not change the expected score to first order, so slower convergence of ML in the first stage does not propagate strongly into the second stage [2510.23534]. A standard DML estimator then solves
\[
\frac{1}{n}\sum_{i=1}^n \psi(W_i;\widehat\eta,\widehat\theta)=0,
\]
or its cross‑fitted analogue [2105.15197], [1701.08687].

A second foundational object is the **Riesz representer** \(\alpha_0\), defined by
\[
E[m(W,\gamma)] = E[\alpha_0(W)\gamma(W)]
\]
or, in the notation of one paper,
\[
E[m(W,\gamma)] = E[v_m(X)\gamma(X)]\quad \forall \gamma,
\]
with \(\alpha_0=v_m\) [2510.23534], [2105.15197]. Under mean-square continuity, the Riesz representation theorem guarantees existence of such an object, and this is the nuisance that converts an abstract linear functional into an orthogonal score [2105.15197]. In many examples \(\alpha_0\) becomes a propensity-score expression, a density ratio, or a score of a joint density [2510.23534].

This suggests a unifying viewpoint: DML is not only an inference procedure after nuisance estimation, but also a way of organizing the nuisance problem itself around the orthogonal score and the Riesz map [2510.23534].

## 2. Score construction, double robustness, and cross-fitting

For binary treatment effects, the standard efficient score for the average treatment effect is the augmented inverse probability weighted score
\[
h^{\mathrm{AIPW}}(W;\eta)
=
\Big(\frac{D}{\pi(Z)}-\frac{1-D}{1-\pi(Z)}\Big)\{Y-\gamma(X)\}
+\gamma(1,Z)-\gamma(0,Z),
\]
and the orthogonal score is
\[
\psi(W;\eta,\tau)=h^{\mathrm{AIPW}}(W;\eta)-\tau
\]
[2510.23534]. For the average treatment effect on the treated,
\[
m^{\mathrm{ATT}}(W,\gamma) = \frac{D}{p}\{\gamma(1,Z)-\gamma(0,Z)\},
\quad p=\Pr(D=1),
\]
with corresponding orthogonal score
\[
\psi^{\mathrm{ATT}}(W;\eta,\tau)
= m^{\mathrm{ATT}}(W,\gamma)+\alpha^{\mathrm{ATT}}(X)\{Y-\gamma(X)\}-\tau
\]
[2510.23534]. In the partially linear regression formulation, the orthogonalized score can be written as
\[
\psi(Z;\beta,g,m)
= \bigl(Y - g(X_c) - \beta (W - m(X_c))\bigr)\,(W - m(X_c)),
\]
which yields the familiar residual-on-residual regression after partialling out \(X_c\) from both \(Y\) and \(W\) [2403.14385].

These scores are **doubly robust** in the usual semiparametric sense. One finite-sample treatment states
\[
\theta_0 = E\bigl[m(W,\gamma_0)+\alpha(W)\{Y-\gamma_0(W)\}\bigr]
=E\bigl[m(W,\gamma)+\alpha_0^{\min}(W)\{Y-\gamma(W)\}\bigr]
\]
for all \(\gamma\) and \(\alpha\) in the relevant spaces [2105.15197]. In words, identification of the target is robust if either the regression nuisance is correct or the Riesz representer is correct. Some recent work also isolates a further property, termed *double robustness to ill-posedness*, where projected errors can replace ordinary mean-square errors in inverse problems [2105.15197].

Cross-fitting is the operational complement to orthogonality. The generic algorithm partitions the sample into folds \(I_\ell\), estimates nuisance functions \(\hat\gamma_\ell\) and \(\hat\alpha_\ell\) using only \(I_\ell^c\), and computes
\[
\hat{\theta}
=
\frac{1}{n}\sum_{\ell=1}^L\sum_{i\in I_\ell}
\bigl[m(W_i,\hat{\gamma}_\ell)+\hat{\alpha}_\ell(W_i)\{Y_i-\hat{\gamma}_\ell(W_i)\}\bigr]
\]
[2105.15197]. In the treatment-effect note built around ATE and ATTE, the final estimator is
\[
\tilde\theta_0 = \frac{1}{K} \sum_{k=1}^K \check\theta_0(I_k,I_k^c),
\]
where each fold-specific estimate solves the score equation on a held‑out fold [1701.08687]. Cross-fitting avoids Donsker restrictions, controls overfitting bias, and makes the influence-function expansion exact in terms of learning rates [2105.15197].

A useful implication is that DML separates two roles. Flexible ML enters only through nuisance prediction, while inference is driven by the orthogonal score. This suggests why the same score architecture reappears in treatment effects, average derivatives, density-ratio problems, panel models, and time-series impulse responses [2605.17910], [2411.10009].

## 3. Riesz representers, generalized Riesz regression, and direct DML

A major recent development reframes nuisance estimation itself as a score-targeted problem. “Direct Debiased Machine Learning via Bregman Divergence Minimization” proposes **Direct Debiased Machine Learning (DDML)**, whose core idea is to estimate nuisance parameters by directly targeting the oracle Neyman score via minimizing a Bregman divergence between the score with true nuisances and the score with candidate nuisances [2510.23534].

The framework begins from the score discrepancy
\[
\Delta(W;\eta,\theta^*)
\coloneqq \psi(W;\eta_0,\theta_0) - \psi(W;\eta,\theta^*).
\]
For the Riesz component, it introduces a differentiable and strictly convex generator \(g\) and the pointwise Bregman divergence
\[
\mathrm{BR}^\dagger_g\big(\alpha_0(x)\mid\alpha(x)\big)
= g(\alpha_0(x)) - g(\alpha(x)) - \partial g(\alpha(x))\{\alpha_0(x)-\alpha(x)\}.
\]
Using linearity of \(m\), one obtains the observable population objective
\[
\mathrm{BR}_g(\alpha)
= E\big[-g(\alpha(X)) + \partial g(\alpha(X))\alpha(X) - m\{\partial g(\alpha)\}\big],
\]
and **generalized Riesz regression** is
\[
\alpha^* \coloneqq \arg\min_{\alpha\in\mathcal A}\mathrm{BR}_g(\alpha)
\]
[2510.23534]. Empirically,
\[
\widehat{\alpha}_n
\coloneqq \arg\min_{\alpha\in\mathcal A}\widehat{\mathrm{BR}_g(\alpha)} + \lambda J(\alpha),
\]
with
\[
\widehat{\mathrm{BR}_g(\alpha)}
=
\frac{1}{n}\sum_{i=1}^n
\big\{-g(\alpha(X_i)) + \partial g(\alpha(X_i))\alpha(X_i) - m\{\partial g(\alpha)\}(W_i)\big\}.
\]

This construction unifies several previously separate literatures. With squared loss, generalized Riesz regression recovers classical Riesz regression and least-squares importance fitting. With KL-type losses it recovers entropy balancing, KLIEP-style density-ratio estimators, and related calibrated estimators [2510.23534]. A later software paper presents this same framework as “generalized Riesz regression, a unified framework for estimating Riesz representers by minimizing empirical Bregman divergences,” and explicitly notes that covariate balancing, nearest-neighbor matching, calibrated estimation, and density ratio estimation arise as special cases [2602.17543].

A central software principle is **automatic regressor balancing (ARB)**. Given a Bregman generator \(g\) and a representer model \(\alpha_\beta(x)=\zeta^{-1}(x,\phi(x)'\beta)\), the package sets the link
\[
\zeta(x,\alpha)=\partial_\alpha g(x,\alpha),
\]
so the dual coordinate is linear in \(\beta\) [2602.17543]. The KKT conditions then imply balancing equations
\[
\frac{1}{n}\sum_{i=1}^n \bigl[\widehat{\alpha}(X_i)\phi_j(X_i) - m(W_i,\phi_j)\bigr] + \lambda s_j = 0,
\]
and, without regularization,
\[
\frac{1}{n}\sum_{i=1}^n \widehat{\alpha}(X_i)\phi_j(X_i)
=
\frac{1}{n}\sum_{i=1}^n m(W_i,\phi_j)
\]
[2602.17543]. In treatment-effect applications, this is exactly the covariate balancing property.

This suggests a general synthesis. Classical DML treats the regression function and the Riesz representer as separate nuisances estimated in separate steps. DDML and generalized Riesz regression instead organize nuisance learning around the discrepancy in the orthogonal score itself, thereby connecting automatic debiased ML, covariate balancing, TMLE, and density-ratio estimation within a single convex-analytic template [2510.23534].

## 4. Finite-sample guarantees, rates, and efficiency

The most explicit finite-sample account in the supplied literature formulates DML as a theorem with Gaussian approximation guarantees for arbitrary global and local functionals [2105.15197]. Under mean square continuity,
\[
E\{m(W,\gamma)^2\}\le \bar Q \bigl(E\{\gamma(W)^2\}\bigr)^q,
\]
there exists a minimal Riesz representer \(\alpha_0^{\min}\), and the oracle score is
\[
\psi_0(W)=m(W,\gamma_0)+\alpha_0^{\min}(W)\{Y-\gamma_0(W)\}-\theta_0
\]
[2105.15197]. The central result gives a finite-sample Berry–Esseen-style bound:
\[
\sup_{z\in\mathbb{R}}
\left|
\text{pr}\!\left\{\frac{\sqrt{n}}{\sigma}(\hat{\theta}-\theta_0)\le z\right\}
-\Phi(z)
\right|
\le
c^{BE}\left(\frac{\kappa}{\sigma}\right)^3 n^{-1/2}
+\frac{\Delta}{\sqrt{2\pi}+\epsilon},
\]
where \(\Delta\) depends on nuisance mean-square errors and projected errors [2105.15197]. The rate is \(n^{-1/2}\) for global functionals, and it degrades gracefully for local functionals.

The asymptotic message across the DML literature is consistent. If nuisance estimates satisfy product-rate conditions such as
\[
\|\widehat{\alpha}-\alpha_0\|_{L_2(P)}\,
\|\widehat{\gamma}-\gamma_0\|_{L_2(P)} = o_p(n^{-1/2}),
\]
or the stronger but familiar symmetric condition that both \(L_2\)-errors are \(o_p(n^{-1/4})\), then the cross‑fitted DML estimator is root‑\(n\) consistent and asymptotically normal [2510.23534], [2105.15197], [1701.08687]. One summary statement is
\[
\sqrt{n}(\widehat\theta-\theta_0)\overset{d}{\to}\mathcal{N}(0,V),
\]
with \(V\) equal to the semiparametric efficiency bound when the score is the efficient influence function [2510.23534].

Several papers adapt this structure beyond standard i.i.d. treatment effects. For continuous treatments, kernel-based DML estimators of the dose-response function satisfy
\[
\sqrt{nh^{d_T}\big(\hat\beta_t - \beta_t - h^2\mathsf{B}_t\big)\overset{d}\longrightarrow \mathcal{N}(0,\mathsf{V}_t),
\]
and partial effects satisfy
\[
\sqrt{nh^{d_T+2}\big(\hat\theta_t - \theta_t - h^2\mathsf{B}_t^\theta\big)
\overset{d}\longrightarrow \mathcal{N}(0,\mathsf{V}_t^\theta)
\]
[2004.03036]. For mediation with continuous treatments, the mediated response estimator obeys
\[
\sqrt{n h} \left(\hat\eta_{t,t'} - \eta_{t,t'} - h^2 B_{t,t'}\right) \xrightarrow{d} \mathcal N\big(0, V_{t,t'}\big)
\]
under kernel localization and asymptotic Neyman orthogonality [2503.06156]. For time-series impulse responses, the cross‑fitted estimator satisfies
\[
\sqrt{T-k_T}(\hat\theta^{(h)} - \theta_0^{(h)})
\overset{d}{\to} \mathcal{N}(0,V_0^{(h)})
\]
under mixing conditions and blocked cross‑fitting with gaps [2411.10009]. For panel NPIV with endogenous continuous treatments and two-way fixed effects,
\[
\sqrt{N}(\widehat{\theta}_{0t}^d(s) - \theta_{0t}(s)) \overset{d}{\to} \mathcal{N}(0, \Psi)
\]
with a plug‑in variance estimator [2605.17910].

A plausible implication is that DML is best understood not as a single estimator, but as a family of orthogonal score constructions whose asymptotic scale depends on the regularity of the target: parametric for global low‑dimensional functionals, slower for localized or kernelized functionals, but still driven by the same orthogonality logic.

## 5. Relations to TMLE, localized DML, covariate balancing, and unobserved heterogeneity

One of the most striking features of recent work is how many seemingly distinct procedures can be rewritten as DML variants or special cases of orthogonal-score design.

**TMLE.** In the DDML framework, the regression nuisance can be updated through a TMLE-style fluctuation. Starting from \(\widehat\gamma^{(0)}\) and estimated \(\widehat\alpha\), solve
\[
\sum_{i=1}^n \{Y_i - (\widehat\gamma^{(0)}(X_i)+\epsilon\widehat\alpha(X_i))\}\widehat\alpha(X_i)=0
\]
to obtain
\[
\widehat\epsilon
=
\frac{\sum_i \widehat\alpha(X_i)\{Y_i-\widehat\gamma^{(0)}(X_i)\}}
{\sum_i \widehat\alpha(X_i)^2},
\quad
\widehat\gamma^{(1)}(x)=\widehat\gamma^{(0)}(x)+\widehat\epsilon\,\widehat\alpha(x),
\]
and then
\[
\widehat\theta^{\text{TMLE}}
=\frac1n\sum_{i=1}^n m(X_i,\widehat\gamma^{(1)}).
\]
This is “exactly the TMLE clever covariate update in the linear fluctuation submodel,” here derived as a special case of Neyman targeted estimation [2510.23534].

**Localized DML.** When nuisances depend on the parameter itself, as in quantile treatment effects, standard DML would require estimating an entire nuisance family. “Localized Debiased Machine Learning” instead estimates nuisances only at a rough initial value \(\tilde\theta\), then solves the orthogonal equation with those localized nuisances [1912.12945]. The method yields the same first-order behavior as the infeasible oracle while requiring only ordinary regression or classification tasks, not full conditional CDF estimation [1912.12945].

**Covariate balancing.** In treatment effect estimation, the Riesz representer often equals an inverse propensity weight. Under linear models and squared-loss generalized Riesz regression, the dual problem becomes a stable balancing problem with moment constraints
\[
\sum_i \widehat\alpha_i\Phi(D_i,Z_i)=\sum_i\{\Phi(1,Z_i)-\Phi(0,Z_i)\},
\]
and under logistic models with tailored KL losses one recovers entropy balancing constraints
\[
\sum_i \mathbb{1}[D_i=1]w_i\Phi(Z_i)=\sum_i \mathbb{1}[D_i=0]w_i\Phi(Z_i)
\]
[2510.23534]. Recent software work packages this under the label automatic regressor balancing [2602.17543].

**Unobserved heterogeneity.** A 2025 paper extends DML to models with nonparametric unobserved heterogeneity and partial identification. Its central characterization states that a moment \(g\) is orthogonal for a target \(\psi\) if and only if there exists a constant \(c\) such that
\[
\mathbb E[g(Z,\lambda_0)\mid \alpha,X]=c\,r_\eta(\alpha,X,\psi_0,\theta_0),
\quad
S_\theta^* g = c\,r_\theta,
\]
and \(g\) is relevant iff \(c\neq 0\) [2507.13788]. Under additional support conditions, the resulting moments are globally robust to the distribution of the unobserved heterogeneity. This extends DML beyond settings where the nuisance is merely high-dimensional and into cases where it is itself a nonparametric latent distribution [2507.13788].

These connections clarify a common misconception. DML is not synonymous with one partially linear algorithm or one AIPW formula. The orthogonal-score principle encompasses TMLE-style targeting, balancing weights, localized one-step corrections, automatic Riesz learning, and certain latent-variable constructions when the relevant orthogonal moments exist [2510.23534], [1912.12945], [2507.13788].

## 6. Applications, limitations, and current directions

The application range in the supplied papers is broad. Continuous-treatment dose-response estimation uses kernel-based doubly robust moments and cross-fitting [2004.03036]. Mediation analysis with continuous treatments targets the mediated response curve and derives direct and indirect effects from it [2503.06156]. A panel NPIV framework handles two-way fixed effects, endogenous continuous treatments, dynamic panels, and average derivative effects through orthogonal scores and a penalized GMM estimate of the Riesz representer [2605.17910]. Time-series DML estimates impulse response functions by treating them as average treatment effects in a single stochastic process and combining blocked cross-fitting with HAC variance estimation [2411.10009]. Applied evaluations show how DML with flexible first stages can improve adjustment for nonlinear confounding in observational studies [2403.14385], and one transportation application uses a partially linear DML model to estimate the causal effect of traffic density on pedestrian waiting time [2212.11322].

Several practical patterns recur. First, flexible nuisance estimation matters. In a broad empirical evaluation, lasso in DML with untransformed variables often behaves much like linear regression, whereas flexible tree‑based methods, GAMs, and neural networks better handle nonlinear confounding [2403.14385]. Second, cross-fitting is repeatedly recommended, including in complex settings such as two‑way fixed effects and time series where specialized fold construction is required [2605.17910], [2411.10009]. Third, the choice of loss for Riesz learning affects stability: squared loss may be sensitive to large ratios, while KL or power-divergence losses can trade efficiency for robustness [2510.23534].

At the same time, the literature emphasizes that DML does not replace causal identification. One application-oriented review states that DML relaxes functional form assumptions but continues to critically depend on standard assumptions about causal structure and identification [2403.14385]. Unobserved confounding, collider adjustment, weak overlap, and invalid instruments remain substantive threats [2403.14385], [2212.11322]. In teacher value-added models, some policy-relevant functionals such as CDFs and quantiles do not admit relevant orthogonal moments, so first-order regularization bias cannot be eliminated through the DML‑UH strategy [2507.13788].

A further recent direction is **anytime-valid inference**. One paper strengthens standard DML conditions slightly to obtain time-uniform confidence sequences for causal parameters at arbitrary stopping times, thereby extending DML from fixed‑\(n\) asymptotics to sequential inference [2408.09598]. Another direction is software automation: the `genriesz` package exposes target linear functionals through a black-box oracle and returns RA, RW, ARW, and TMLE-style estimators with cross-fitting, confidence intervals, and \(p\)-values [2602.17543].

The overall picture is that Debiased Machine Learning is a general framework for obtaining valid inference on low-dimensional targets when nuisance structure is estimated flexibly. Its core ingredients are Riesz representation, orthogonal scores, and cross-fitting. Recent work suggests a broader interpretation in which nuisance estimation itself becomes a score-targeted optimization problem, often expressible through Bregman divergences, balancing conditions, and targeted updating. This suggests that the conceptual center of DML is not a particular estimator, but the design of moments that preserve first-order validity under flexible, regularized learning [2510.23534], [2105.15197].

Source: https://www.emergentmind.com/topics/debiased-machine-learning-dml