Semiparametric doubly robust targeted double machine learning: a review (2203.06469v2)

Abstract: In this review we cover the basics of efficient nonparametric parameter estimation (also called functional estimation), with a focus on parameters that arise in causal inference problems. We review both efficiency bounds (i.e., what is the best possible performance for estimating a given parameter?) and the analysis of particular estimators (i.e., what is this estimator's error, and does it attain the efficiency bound?) under weak assumptions. We emphasize minimax-style efficiency bounds, worked examples, and practical shortcuts for easing derivations. We gloss over most technical details, in the interest of highlighting important concepts and providing intuition for main ideas.

• The paper introduces semiparametric efficiency and doubly robust estimation to achieve √n-consistent causal effect estimators.
• It employs efficient influence functions with one-step bias corrections and cross-fitting to enhance estimator accuracy in high-dimensional settings.
• The review offers practical insights into integrating machine learning with causal inference and outlines future research directions.

Overview of Semiparametric Doubly Robust Targeted Double Machine Learning

This review paper explores the domain of efficient nonparametric parameter estimation with a specific focus on causal inference applications. It presents a structured approach to achieving and assessing the performance of estimators for certain functional parameters. The central theme of the paper is semiparametric efficiency, doubly robust estimation, and the utilization of machine learning techniques aimed at minimizing estimation error under weak assumptions.

The paper first establishes the groundwork for efficient parameter estimation by discussing various functionals, particularly those relevant to causal inference, such as the average treatment effect (ATE), variance-weighted treatment effects, and stochastic intervention effects. Each functional has unique properties that lend themselves to different estimation methods and efficiency considerations. The author emphasizes achieving $\sqrt{n}$-consistent and asymptotically normal estimators, highlighting the contrast with nonparametric regression or density estimation, where such rates are typically unobtainable without the imposition of strong assumptions.

Efficient influence functions (EIFs) are identified as central tools for characterizing the difficulties associated with estimating functionals. These EIFs enable the derivation of nonparametric efficiency bounds, akin to the Cramer-Rao bound in parametric models, and facilitate the development of targeted estimation strategies. The methodological focus of the paper is on bias correction, exemplified by one-step estimators, which integrate influence function-based corrections to improve the asymptotic properties of plug-in estimators.

The utilization of doubly robust estimators is thoroughly explored, illustrating how efficiency can be achieved when either of two nuisance parameters (e.g., propensity scores and outcome regressions) are consistently estimated. This is particularly relevant in the context of high-dimensional or flexible machine learning approaches, which may fail to be Donsker. The paper suggests adopting sample splitting or cross-fitting methods as pragmatic alternatives to mitigate potential overfitting problems without sacrificing asymptotic efficiency.

The paper also addresses challenges and solutions in applying semiparametric methods to a broad array of functionals, showcasing through comprehensive examples how these techniques can yield robust, efficient estimators in practical applications. The author provides detailed derivations of von Mises expansions and shows how to compute influence functions either through traditional approaches or newer, more intuitive methods like Gateaux derivatives.

Theoretical implications extend beyond practical estimation, offering insights into the interplay between functional smoothness, model sparsity, and achievable estimation bounds in complex data settings. Additionally, it raises the potential for broader application of these semiparametric techniques to other domains where causal inference is critical.

In summarizing practical developments, the review prioritizes understanding the implications of efficiency bounds and highlighting situations where certain estimation procedures fail to achieve optimal rates, thereby advocating for continued development in high-dimensional settings where traditional approaches fall short. The discussion on non-pathwise differentiable functionals also points to future research directions, particularly regarding functionals incorporating non-differentiable transformations of nuisance parameters or those defined over infinite-dimensional spaces.

In essence, the paper serves as both a comprehensive review and a compelling endorsement of advanced estimation frameworks that leverage semiparametric methods, influence functions, and modern machine learning techniques to effectively estimate causal parameters under challenging conditions.