Double/Debiased Machine Learning for Treatment and Causal Parameters

Abstract: Most modern supervised statistical/ML methods are explicitly designed to solve prediction problems very well. Achieving this goal does not imply that these methods automatically deliver good estimators of causal parameters. Examples of such parameters include individual regression coefficients, average treatment effects, average lifts, and demand or supply elasticities. In fact, estimates of such causal parameters obtained via naively plugging ML estimators into estimating equations for such parameters can behave very poorly due to the regularization bias. Fortunately, this regularization bias can be removed by solving auxiliary prediction problems via ML tools. Specifically, we can form an orthogonal score for the target low-dimensional parameter by combining auxiliary and main ML predictions. The score is then used to build a de-biased estimator of the target parameter which typically will converge at the fastest possible 1/root(n) rate and be approximately unbiased and normal, and from which valid confidence intervals for these parameters of interest may be constructed. The resulting method thus could be called a "double ML" method because it relies on estimating primary and auxiliary predictive models. In order to avoid overfitting, our construction also makes use of the K-fold sample splitting, which we call cross-fitting. This allows us to use a very broad set of ML predictive methods in solving the auxiliary and main prediction problems, such as random forest, lasso, ridge, deep neural nets, boosted trees, as well as various hybrids and aggregators of these methods.

• The paper introduces Double/Debiased Machine Learning using Neyman orthogonal scores and cross-fitting to robustly estimate low-dimensional treatment effects.
• It leverages weak assumptions across various machine learning techniques to reduce regularization bias and overfitting in high-dimensional settings.
• Empirical applications demonstrate its practical utility in estimating causal effects in economic studies such as unemployment insurance impacts and asset accumulation.

Overview of Double/Debiased Machine Learning for Treatment and Structural Parameters

The paper revisits the semiparametric estimation problem where inference is made on a low-dimensional parameter while accounting for high-dimensional nuisance parameters. The authors propose methods termed as Double/Debiased Machine Learning (DML) that leverage machine learning techniques to estimate nuisance parameters in modern high-dimensional settings where traditional assumptions break down. This approach circumvents issues such as regularization bias and overfitting by utilizing critical ingredients such as Neyman orthogonal moments, which provide insensitivity to nuisance parameters, and cross-fitting, a form of sample splitting for efficient estimation.

Core Contributions

1. Orthogonality and Cross-Fitting: The paper introduces the Neyman orthogonal scores which minimize sensitivity to nuisance parameters. By employing orthogonality, estimators of interest become robust to misspecifications in nuisance parameter estimations. Additionally, cross-fitting—a technique involving partitioning data into disjoint subsets—is used to alleviate any overfitting biases, ensuring that estimation errors do not adversely bias the target parameter estimators.
2. Statistical Theory and Implementation: The authors present a statistical theory for DML that requires only weak assumptions, making it compatible with a wide array of machine learning methods such as lasso, random forests, boosted trees, and neural networks. Furthermore, they present practical implementations for various models including partially linear regression, treatment effect models under unconfoundedness, and instrumental variable models.
3. Empirical Application: The versatility of DML is demonstrated through empirical examples, including the analysis of the effects of unemployment insurance bonus on unemployment duration, the causal effect of 401(k) participation and eligibility on accumulated assets, and the impact of institutions on economic growth.

Theoretical and Practical Implications

The theoretical foundations laid by DML ensure that it provides approximately unbiased and normally distributed point estimators that are consistent and facilitate valid confidence interval construction. The practical implementation of these methodologies allows researchers to tackle complex estimation problems that involve high-dimensionality, thereby broadening the application scope of econometric analysis in modern datasets.

Future Directions

While the methodological framework is robust, expanding its possibilities towards optimizing efficiency and further reducing computational complexities could be insightful. There exists potential for extending these techniques to incorporate more dynamic datasets and models, potentially offering refined approaches to causal inference in machine learning domains.

By circumventing traditional restrictions on nuisance parameter complexity, DML provides a robust and adaptable framework for empirical research, offering promising avenues for leveraging machine learning for rigorous statistical inference.