A joint test of unconfoundedness and common trends (2404.16961v3)

Abstract: This paper introduces an overidentification test of two alternative assumptions to identify the average treatment effect on the treated in a two-period panel data setting: unconfoundedness and common trends. Under the unconfoundedness assumption, treatment assignment and post-treatment outcomes are independent, conditional on control variables and pre-treatment outcomes, which motivates including pre-treatment outcomes in the set of controls. Conversely, under the common trends assumption, the trend and the treatment assignment are independent, conditional on control variables. This motivates employing a Difference-in-Differences (DiD) approach by comparing the differences between pre- and post-treatment outcomes of the treatment and control group. Given the non-nested nature of these assumptions and their often ambiguous plausibility in empirical settings, we propose a joint test using a doubly robust statistic that can be combined with machine learning to control for observed confounders in a data-driven manner. We discuss various causal models that imply the satisfaction of either common trends, unconfoundedness, or both assumptions jointly, and we investigate the finite sample properties of our test through a simulation study. Additionally, we apply the proposed method to five empirical examples using publicly available datasets and find the test to reject the null hypothesis in two cases.

• The paper introduces a novel joint test for the crucial causal inference assumptions of unconfoundedness and common trends in two-period panel data analysis.
• It proposes a flexible doubly robust statistic augmented with machine learning to estimate confounder effects and provide robustness against model misspecification.
• Empirical validation shows the test's effectiveness in identifying violations of these assumptions, offering researchers a robust tool for validating causal inferences.

Overview of a Joint Test for Unconfoundedness and Common Trends

This paper introduces a methodological framework for the overidentification test of two fundamental assumptions—unconfoundedness and common trends—to identify the average treatment effect on the treated (ATET) in a two-period panel data setting. The discussion delineates the assumptions underlying two prevalent techniques in treatment effect estimation: inclusion of pre-treatment outcomes as controls and the Difference-in-Differences (DiD) approach. It also addresses the conceptual non-nested nature of these assumptions, presenting a novel joint test leveraging a doubly robust statistic, augmented with machine learning for enhanced data-driven control of confounders.

Key Conceptual Foundations

1. Unconfoundedness: The unconfoundedness assumption posits that conditional on control variables and pre-treatment outcomes, the treatment assignment is independent of post-treatment outcomes. This assumption underpins using pre-treatment outcomes as controls in causal inference.
2. Common Trends: The common trends assumption underlies the DiD approach. It assumes that, conditioned on covariates, treatment assignment is independent of the time trends in outcomes, which facilitates leveraging changes in pre- and post-treatment outcomes among treatment and control groups to identify the ATET.

In empirical research, determining which assumption is more plausible is not always clear, motivating the need for a robust testing methodology. This paper fills this gap by proposing a doubly robust testing procedure.

Methodological Contributions

The central contribution of the paper is the development of a joint test for the unconfoundedness and common trends assumptions using a flexible doubly robust statistic that can be coupled with machine learning to estimate confounder effects. The proposed doubly robust estimator provides consistent estimates of ATET when either the outcome models or the propensity scores are correctly specified, providing robustness against misspecifications.

• Doubly Robust Statistics: The paper emphasizes the orthogonality properties of the doubly robust statistic, ensuring asymptotic normality and consistency for large samples even when nuisance parameters are machine-learned. This facilitates the robust estimation of test statistics in high-dimensional settings.
• Implementation with Machine Learning (ML): The methodology leverages ML-based estimators for conditional expectation functions and propensity scores, increasing the adaptability and efficacy of controlling confounding variables.

Empirical Assessment

The paper evaluates the finite-sample properties of the proposed test through simulation studies and applies the method to real-world datasets, including publicly available datasets with different experimental designs and assumptions about treatment assignments. These practical applications validate the effectiveness of the method, demonstrating rejection of the null hypothesis in specific cases, signifying violations of either or both assumptions.

Implications and Future Directions

The results hold significant implications for empirical research, suggesting a more nuanced and robust means of validating treatment effect estimation when underlying assumptions are ambiguous. The methodological framework presented could guide empirical analysts in a variety of fields—from economics to epidemiology—in strengthening causal inferences drawn from panel data.

Future research directions proposed include extending this framework to multi-period data, facilitating more comprehensive analyses of policy interventions and natural experiments. Additionally, further integration with advanced machine learning methods could enhance the flexibility and scalability of the approach in processing increasingly complex datasets.

In summary, this paper provides a rigorous and adaptable framework for improving causal inference in panel data, advocating for empirically grounded validation of foundational assumptions through the innovative application of joint testing and machine learning techniques.