Debiased Machine Learning (DML)

• Debiased Machine Learning is a framework that estimates low-dimensional causal parameters by constructing orthogonal score functions to offset errors in high-dimensional nuisance estimates.
• It leverages Neyman orthogonality and doubly robust techniques, ensuring that small errors in nuisance estimation minimally affect the target parameter inference.
• Recent advances integrate concepts like Bregman divergence minimization and Riesz representers, unifying various methods such as TMLE and covariate balancing in causal inference.

Searching arXiv for recent and foundational papers on debiased machine learning to ground the article. Debiased Machine Learning (DML) is a framework for estimating low‑dimensional causal and structural parameters in models with high‑dimensional or complex nuisance components by constructing orthogonal (Neyman‑orthogonal, doubly robust) score/moment functions that are insensitive to small errors in nuisance estimation, estimating these nuisance functions using flexible ML methods, and applying cross‑fitting (sample splitting) so that the resulting estimator of the target parameter is root‑$n$ consistent, asymptotically normal, and amenable to valid inference (Chernozhukov et al., 2017). In the formulations emphasized across recent work, DML is also a meta algorithm and a finite-sample theorem for functionals of machine learning predictors, and a broader view treats nuisance estimation itself as a target of score design through Riesz representers, Bregman divergence minimization, covariate balancing, and TMLE‑style updating (Chernozhukov et al., 2021, Kato, 27 Oct 2025).

1. Conceptual foundations

The basic setting is that one observes i.i.d. data $W=(X,Y)$ or $W=(Y,D,Z)$, and the parameter of interest is a scalar functional of an unknown regression function. A canonical representation is

$\theta_0 = E\big[m(W,\gamma_0)\big],$

where $\gamma_0(x)=E[Y\mid X=x]$ and $m$ is linear in $\gamma$ (Kato, 27 Oct 2025, Chernozhukov et al., 2021). In treatment-effect settings, examples include the average treatment effect

$\theta_0 = E[\gamma_0(1,V,X) - \gamma_0(0,V,X)],$

while for local functionals one considers kernel‑weighted limits such as

$$\theta_0^{\lim} = \lim_{h\to 0} E[\ell_h(W_j)m(W,\gamma_0)].$$

The common difficulty is that naïvely plugging a flexible ML estimate $\widehat\gamma$ into the identifying functional can yield poor performance because ML estimators converge slower than $W=(X,Y)$0 and the functional is not “smooth” enough (Kato, 27 Oct 2025, Chernozhukov et al., 2021).

The central device is a Neyman orthogonal score. In the linear-functional framework, the score is

$W=(X,Y)$1

where $W=(X,Y)$2 and $W=(X,Y)$3 is the Riesz representer (Kato, 27 Oct 2025). At the truth,

$W=(X,Y)$4

and for any perturbation direction $W=(X,Y)$5,

$W=(X,Y)$6

This orthogonality means that small errors in $W=(X,Y)$7 do not change the expected score to first order, so slower convergence of ML in the first stage does not propagate strongly into the second stage (Kato, 27 Oct 2025). A standard DML estimator then solves

$W=(X,Y)$8

or its cross‑fitted analogue (Chernozhukov et al., 2021, Chernozhukov et al., 2017).

A second foundational object is the Riesz representer $W=(X,Y)$9, defined by

$W=(Y,D,Z)$0

or, in the notation of one paper,

$W=(Y,D,Z)$1

with $W=(Y,D,Z)$2 (Kato, 27 Oct 2025, Chernozhukov et al., 2021). Under mean-square continuity, the Riesz representation theorem guarantees existence of such an object, and this is the nuisance that converts an abstract linear functional into an orthogonal score (Chernozhukov et al., 2021). In many examples $W=(Y,D,Z)$3 becomes a propensity-score expression, a density ratio, or a score of a joint density (Kato, 27 Oct 2025).

This suggests a unifying viewpoint: DML is not only an inference procedure after nuisance estimation, but also a way of organizing the nuisance problem itself around the orthogonal score and the Riesz map (Kato, 27 Oct 2025).

2. Score construction, double robustness, and cross-fitting

For binary treatment effects, the standard efficient score for the average treatment effect is the augmented inverse probability weighted score

$W=(Y,D,Z)$4

and the orthogonal score is

$W=(Y,D,Z)$5

(Kato, 27 Oct 2025). For the average treatment effect on the treated,

$W=(Y,D,Z)$6

with corresponding orthogonal score

$W=(Y,D,Z)$7

(Kato, 27 Oct 2025). In the partially linear regression formulation, the orthogonalized score can be written as

$W=(Y,D,Z)$8

which yields the familiar residual-on-residual regression after partialling out $W=(Y,D,Z)$9 from both $\theta_0 = E\big[m(W,\gamma_0)\big],$0 and $\theta_0 = E\big[m(W,\gamma_0)\big],$1 (Fuhr et al., 2024).

These scores are doubly robust in the usual semiparametric sense. One finite-sample treatment states

$\theta_0 = E\big[m(W,\gamma_0)\big],$2

for all $\theta_0 = E\big[m(W,\gamma_0)\big],$3 and $\theta_0 = E\big[m(W,\gamma_0)\big],$4 in the relevant spaces (Chernozhukov et al., 2021). In words, identification of the target is robust if either the regression nuisance is correct or the Riesz representer is correct. Some recent work also isolates a further property, termed double robustness to ill-posedness, where projected errors can replace ordinary mean-square errors in inverse problems (Chernozhukov et al., 2021).

Cross-fitting is the operational complement to orthogonality. The generic algorithm partitions the sample into folds $\theta_0 = E\big[m(W,\gamma_0)\big],$5, estimates nuisance functions $\theta_0 = E\big[m(W,\gamma_0)\big],$6 and $\theta_0 = E\big[m(W,\gamma_0)\big],$7 using only $\theta_0 = E\big[m(W,\gamma_0)\big],$8, and computes

$\theta_0 = E\big[m(W,\gamma_0)\big],$9

(Chernozhukov et al., 2021). In the treatment-effect note built around ATE and ATTE, the final estimator is

$\gamma_0(x)=E[Y\mid X=x]$0

where each fold-specific estimate solves the score equation on a held‑out fold (Chernozhukov et al., 2017). Cross-fitting avoids Donsker restrictions, controls overfitting bias, and makes the influence-function expansion exact in terms of learning rates (Chernozhukov et al., 2021).

A useful implication is that DML separates two roles. Flexible ML enters only through nuisance prediction, while inference is driven by the orthogonal score. This suggests why the same score architecture reappears in treatment effects, average derivatives, density-ratio problems, panel models, and time-series impulse responses (Wu et al., 18 May 2026, Ballinari et al., 2024).

3. Riesz representers, generalized Riesz regression, and direct DML

A major recent development reframes nuisance estimation itself as a score-targeted problem. “Direct Debiased Machine Learning via Bregman Divergence Minimization” proposes Direct Debiased Machine Learning (DDML), whose core idea is to estimate nuisance parameters by directly targeting the oracle Neyman score via minimizing a Bregman divergence between the score with true nuisances and the score with candidate nuisances (Kato, 27 Oct 2025).

The framework begins from the score discrepancy

$\gamma_0(x)=E[Y\mid X=x]$1

For the Riesz component, it introduces a differentiable and strictly convex generator $\gamma_0(x)=E[Y\mid X=x]$2 and the pointwise Bregman divergence

$\gamma_0(x)=E[Y\mid X=x]$3

Using linearity of $\gamma_0(x)=E[Y\mid X=x]$4, one obtains the observable population objective

$\gamma_0(x)=E[Y\mid X=x]$5

and generalized Riesz regression is

$\gamma_0(x)=E[Y\mid X=x]$6

(Kato, 27 Oct 2025). Empirically,

$\gamma_0(x)=E[Y\mid X=x]$7

with

$\gamma_0(x)=E[Y\mid X=x]$8

This construction unifies several previously separate literatures. With squared loss, generalized Riesz regression recovers classical Riesz regression and least-squares importance fitting. With KL-type losses it recovers entropy balancing, KLIEP-style density-ratio estimators, and related calibrated estimators (Kato, 27 Oct 2025). A later software paper presents this same framework as “generalized Riesz regression, a unified framework for estimating Riesz representers by minimizing empirical Bregman divergences,” and explicitly notes that covariate balancing, nearest-neighbor matching, calibrated estimation, and density ratio estimation arise as special cases (Kato, 19 Feb 2026).

A central software principle is automatic regressor balancing (ARB). Given a Bregman generator $\gamma_0(x)=E[Y\mid X=x]$9 and a representer model $m$0, the package sets the link

$m$1

so the dual coordinate is linear in $m$2 (Kato, 19 Feb 2026). The KKT conditions then imply balancing equations

$m$3

and, without regularization,

$m$4

(Kato, 19 Feb 2026). In treatment-effect applications, this is exactly the covariate balancing property.

This suggests a general synthesis. Classical DML treats the regression function and the Riesz representer as separate nuisances estimated in separate steps. DDML and generalized Riesz regression instead organize nuisance learning around the discrepancy in the orthogonal score itself, thereby connecting automatic debiased ML, covariate balancing, TMLE, and density-ratio estimation within a single convex-analytic template (Kato, 27 Oct 2025).

4. Finite-sample guarantees, rates, and efficiency

The most explicit finite-sample account in the supplied literature formulates DML as a theorem with Gaussian approximation guarantees for arbitrary global and local functionals (Chernozhukov et al., 2021). Under mean square continuity,

$m$5

there exists a minimal Riesz representer $m$6, and the oracle score is

$m$7

(Chernozhukov et al., 2021). The central result gives a finite-sample Berry–Esseen-style bound: $m$8 where $m$9 depends on nuisance mean-square errors and projected errors (Chernozhukov et al., 2021). The rate is $\gamma$0 for global functionals, and it degrades gracefully for local functionals.

The asymptotic message across the DML literature is consistent. If nuisance estimates satisfy product-rate conditions such as

$\gamma$1

or the stronger but familiar symmetric condition that both $\gamma$2-errors are $\gamma$3, then the cross‑fitted DML estimator is root‑$\gamma$4 consistent and asymptotically normal (Kato, 27 Oct 2025, Chernozhukov et al., 2021, Chernozhukov et al., 2017). One summary statement is

$\gamma$5

with $\gamma$6 equal to the semiparametric efficiency bound when the score is the efficient influence function (Kato, 27 Oct 2025).

Several papers adapt this structure beyond standard i.i.d. treatment effects. For continuous treatments, kernel-based DML estimators of the dose-response function satisfy

$\gamma$7

and partial effects satisfy

$\gamma$8

(Colangelo et al., 2020). For mediation with continuous treatments, the mediated response estimator obeys

$\gamma$9

under kernel localization and asymptotic Neyman orthogonality (Zenati et al., 8 Mar 2025). For time-series impulse responses, the cross‑fitted estimator satisfies

$\theta_0 = E[\gamma_0(1,V,X) - \gamma_0(0,V,X)],$0

under mixing conditions and blocked cross‑fitting with gaps (Ballinari et al., 2024). For panel NPIV with endogenous continuous treatments and two-way fixed effects,

$\theta_0 = E[\gamma_0(1,V,X) - \gamma_0(0,V,X)],$1

with a plug‑in variance estimator (Wu et al., 18 May 2026).

A plausible implication is that DML is best understood not as a single estimator, but as a family of orthogonal score constructions whose asymptotic scale depends on the regularity of the target: parametric for global low‑dimensional functionals, slower for localized or kernelized functionals, but still driven by the same orthogonality logic.

5. Relations to TMLE, localized DML, covariate balancing, and unobserved heterogeneity

One of the most striking features of recent work is how many seemingly distinct procedures can be rewritten as DML variants or special cases of orthogonal-score design.

TMLE. In the DDML framework, the regression nuisance can be updated through a TMLE-style fluctuation. Starting from $\theta_0 = E[\gamma_0(1,V,X) - \gamma_0(0,V,X)],$2 and estimated $\theta_0 = E[\gamma_0(1,V,X) - \gamma_0(0,V,X)],$3, solve

$\theta_0 = E[\gamma_0(1,V,X) - \gamma_0(0,V,X)],$4

to obtain

$\theta_0 = E[\gamma_0(1,V,X) - \gamma_0(0,V,X)],$5

and then

$\theta_0 = E[\gamma_0(1,V,X) - \gamma_0(0,V,X)],$6

This is “exactly the TMLE clever covariate update in the linear fluctuation submodel,” here derived as a special case of Neyman targeted estimation (Kato, 27 Oct 2025).

Localized DML. When nuisances depend on the parameter itself, as in quantile treatment effects, standard DML would require estimating an entire nuisance family. “Localized Debiased Machine Learning” instead estimates nuisances only at a rough initial value $\theta_0 = E[\gamma_0(1,V,X) - \gamma_0(0,V,X)],$7, then solves the orthogonal equation with those localized nuisances (Kallus et al., 2019). The method yields the same first-order behavior as the infeasible oracle while requiring only ordinary regression or classification tasks, not full conditional CDF estimation (Kallus et al., 2019).

Covariate balancing. In treatment effect estimation, the Riesz representer often equals an inverse propensity weight. Under linear models and squared-loss generalized Riesz regression, the dual problem becomes a stable balancing problem with moment constraints

$\theta_0 = E[\gamma_0(1,V,X) - \gamma_0(0,V,X)],$8

and under logistic models with tailored KL losses one recovers entropy balancing constraints

$\theta_0 = E[\gamma_0(1,V,X) - \gamma_0(0,V,X)],$9

(Kato, 27 Oct 2025). Recent software work packages this under the label automatic regressor balancing (Kato, 19 Feb 2026).

Unobserved heterogeneity. A 2025 paper extends DML to models with nonparametric unobserved heterogeneity and partial identification. Its central characterization states that a moment $$\theta_0^{\lim} = \lim_{h\to 0} E[\ell_h(W_j)m(W,\gamma_0)].$$0 is orthogonal for a target $$\theta_0^{\lim} = \lim_{h\to 0} E[\ell_h(W_j)m(W,\gamma_0)].$$1 if and only if there exists a constant $$\theta_0^{\lim} = \lim_{h\to 0} E[\ell_h(W_j)m(W,\gamma_0)].$$2 such that

$$\theta_0^{\lim} = \lim_{h\to 0} E[\ell_h(W_j)m(W,\gamma_0)].$$3

and $$\theta_0^{\lim} = \lim_{h\to 0} E[\ell_h(W_j)m(W,\gamma_0)].$$4 is relevant iff $$\theta_0^{\lim} = \lim_{h\to 0} E[\ell_h(W_j)m(W,\gamma_0)].$$5 (Argañaraz et al., 18 Jul 2025). Under additional support conditions, the resulting moments are globally robust to the distribution of the unobserved heterogeneity. This extends DML beyond settings where the nuisance is merely high-dimensional and into cases where it is itself a nonparametric latent distribution (Argañaraz et al., 18 Jul 2025).

These connections clarify a common misconception. DML is not synonymous with one partially linear algorithm or one AIPW formula. The orthogonal-score principle encompasses TMLE-style targeting, balancing weights, localized one-step corrections, automatic Riesz learning, and certain latent-variable constructions when the relevant orthogonal moments exist (Kato, 27 Oct 2025, Kallus et al., 2019, Argañaraz et al., 18 Jul 2025).

6. Applications, limitations, and current directions

The application range in the supplied papers is broad. Continuous-treatment dose-response estimation uses kernel-based doubly robust moments and cross-fitting (Colangelo et al., 2020). Mediation analysis with continuous treatments targets the mediated response curve and derives direct and indirect effects from it (Zenati et al., 8 Mar 2025). A panel NPIV framework handles two-way fixed effects, endogenous continuous treatments, dynamic panels, and average derivative effects through orthogonal scores and a penalized GMM estimate of the Riesz representer (Wu et al., 18 May 2026). Time-series DML estimates impulse response functions by treating them as average treatment effects in a single stochastic process and combining blocked cross-fitting with HAC variance estimation (Ballinari et al., 2024). Applied evaluations show how DML with flexible first stages can improve adjustment for nonlinear confounding in observational studies (Fuhr et al., 2024), and one transportation application uses a partially linear DML model to estimate the causal effect of traffic density on pedestrian waiting time (Kamal et al., 2022).

Several practical patterns recur. First, flexible nuisance estimation matters. In a broad empirical evaluation, lasso in DML with untransformed variables often behaves much like linear regression, whereas flexible tree‑based methods, GAMs, and neural networks better handle nonlinear confounding (Fuhr et al., 2024). Second, cross-fitting is repeatedly recommended, including in complex settings such as two‑way fixed effects and time series where specialized fold construction is required (Wu et al., 18 May 2026, Ballinari et al., 2024). Third, the choice of loss for Riesz learning affects stability: squared loss may be sensitive to large ratios, while KL or power-divergence losses can trade efficiency for robustness (Kato, 27 Oct 2025).

At the same time, the literature emphasizes that DML does not replace causal identification. One application-oriented review states that DML relaxes functional form assumptions but continues to critically depend on standard assumptions about causal structure and identification (Fuhr et al., 2024). Unobserved confounding, collider adjustment, weak overlap, and invalid instruments remain substantive threats (Fuhr et al., 2024, Kamal et al., 2022). In teacher value-added models, some policy-relevant functionals such as CDFs and quantiles do not admit relevant orthogonal moments, so first-order regularization bias cannot be eliminated through the DML‑UH strategy (Argañaraz et al., 18 Jul 2025).

A further recent direction is anytime-valid inference. One paper strengthens standard DML conditions slightly to obtain time-uniform confidence sequences for causal parameters at arbitrary stopping times, thereby extending DML from fixed‑$$\theta_0^{\lim} = \lim_{h\to 0} E[\ell_h(W_j)m(W,\gamma_0)].$$6 asymptotics to sequential inference (Dalal et al., 2024). Another direction is software automation: the genriesz package exposes target linear functionals through a black-box oracle and returns RA, RW, ARW, and TMLE-style estimators with cross-fitting, confidence intervals, and $$\theta_0^{\lim} = \lim_{h\to 0} E[\ell_h(W_j)m(W,\gamma_0)].$$7-values (Kato, 19 Feb 2026).

The overall picture is that Debiased Machine Learning is a general framework for obtaining valid inference on low-dimensional targets when nuisance structure is estimated flexibly. Its core ingredients are Riesz representation, orthogonal scores, and cross-fitting. Recent work suggests a broader interpretation in which nuisance estimation itself becomes a score-targeted optimization problem, often expressible through Bregman divergences, balancing conditions, and targeted updating. This suggests that the conceptual center of DML is not a particular estimator, but the design of moments that preserve first-order validity under flexible, regularized learning (Kato, 27 Oct 2025, Chernozhukov et al., 2021).