Heterogeneity Analysis with Heterogeneous Treatments

Abstract: Analysis of effect heterogeneity at the group level is standard practice in empirical treatment evaluation research. However, treatments analyzed are often aggregates of multiple underlying treatments which are themselves heterogeneous, e.g. different modules of a training program or varying exposures. In these settings, conventional approaches such as comparing (adjusted) differences-in-means across groups can produce misleading conclusions when underlying treatment propensities differ systematically between groups. This paper develops a novel decomposition framework that disentangles contributions of effect heterogeneity and qualitatively distinct components of treatment heterogeneity to observed group-level differences. We propose semiparametric debiased machine learning estimators that are robust to complex treatments and limited overlap. We revisit a widely documented gender gap in training returns of an active labor market policy. The decomposition reveals that it is almost entirely driven by women being treated differently than men and not by heterogeneous returns from identical treatments. In particular, women are disproportionately targeted towards vocational training tracks with lower unconditional returns.