Difference-in-differences with "bad controls"

Abstract: This paper considers difference-in-differences identification strategies when the parallel trends assumption holds after conditioning on covariates that may themselves be affected by the treatment (often referred to as "bad controls"). We show that common approaches such as simply dropping bad controls are often ill-advised and develop two alternative approaches that allow bad controls to function as genuine controls despite being affected by treatment. First, we derive explicit conditions that rationalize conditioning only on pre-treatment values of the bad control, leading naturally to the Callaway and Sant'Anna (2021) estimator with pre-treatment values as covariates. Second, under a covariate unconfoundedness condition, we develop imputation and double/debiased machine learning estimators that recover the average treatment effect on the treated. We extend these results to staggered treatment adoption, provide pre-tests for the identifying assumptions, and apply the methods to study the effects of job displacement on earnings.

• The paper shows that both including and omitting a treatment-affected, outcome-relevant covariate can invalidate DiD, and it identifies the ATT by recovering the covariate’s untreated counterfactual evolution.
• It develops pre-treatment conditioning, nested imputation, doubly robust, and double/debiased machine-learning estimators that extend to staggered adoption and support flexible nuisance-function estimation.
• In a job-displacement application, accounting for occupation-score changes estimates roughly 7% lower earnings, while conventional specifications produce effects about 30–40% larger in absolute magnitude.

Difference-in-Differences with “Bad Controls”

Research problem and central contribution

“Difference-in-differences with ‘bad controls’” (2608.03881) studies identification of treatment effects when conditional parallel trends requires covariates that vary over time and may themselves be affected by treatment. This setting is common in labor economics and other applications. Occupation, industry, union status, health, firm affiliation, and prior earnings may all be predictive of untreated outcome trends while also responding causally to the treatment.

The paper’s central claim is contrary to the standard empirical prescription that post-treatment covariates should simply be omitted. When a time-varying covariate is genuinely relevant for untreated outcome trends, dropping it changes the identifying assumption rather than solving the problem. Conversely, directly conditioning on its observed post-treatment value is generally invalid because the observed covariate for treated units corresponds to $X_{t^*}(1)$, whereas conditional parallel trends requires the untreated potential covariate $X_{t^*}(0)$.

The paper formalizes this tension and develops two identification strategies that treat the covariate as both treatment-affected and substantively relevant for outcome trends. The first strategy establishes conditions under which conditioning on the pre-treatment value of the bad control is sufficient. The second allows additional confounders in the evolution of the bad control and identifies the ATT through nested conditional expectations, imputation, doubly robust estimation, and double/debiased machine learning (DDML).

Formal definition and identification failure

The analysis begins with a two-period design. Let $D$ denote treatment status, $Y_t$ the outcome, $X_t$ the potentially treatment-affected covariate, and $Z$ additional exogenous covariates. The target parameter is

$ATT = E[Y_{t^*}(1)-Y_{t^*}(0)\mid D=1].$

The paper defines $X_t$ as a bad control when two properties jointly hold. First, it must be outcome-relevant: conditional on other covariates, untreated outcome changes depend on $X_{t^*}(0)$. Second, treatment must alter the distribution of the covariate for treated units. Thus, the problem is not merely that $X_{t^*}$ is post-treatment; it is that the covariate is simultaneously needed for conditional parallel trends and causally modified by treatment.

The identifying parallel-trends condition is stated in terms of the untreated potential covariate:

$X_{t^*}(0)$0

Under this condition, the ATT requires averaging the untreated comparison-group trend over the distribution of $X_{t^*}(0)$1 for treated units. That distribution is unobserved because treated units reveal $X_{t^*}(0)$2 rather than $X_{t^*}(0)$3. Consequently, conditional parallel trends alone does not identify the ATT when the conditioning covariate is a bad control.

This missing-distribution problem clarifies why both conventional approaches are difficult to justify. Directly using $X_{t^*}(0)$4 replaces the counterfactual distribution of $X_{t^*}(0)$5 among treated units with the observed distribution of $X_{t^*}(0)$6. The resulting estimand is biased whenever treatment changes the covariate in a way that is related to untreated outcome trends. Omitting $X_{t^*}(0)$7 instead imposes a revised parallel-trends condition in which untreated outcome trends depend only on $X_{t^*}(0)$8. That assumption is inconsistent with the premise that $X_{t^*}(0)$9 is an outcome-relevant control.

The distinction is therefore not between “controlling” and “not controlling.” It is between conditioning on the correct counterfactual covariate distribution and conditioning on an observed post-treatment realization.

Two identification strategies

Conditioning on pre-treatment bad controls

The first strategy imposes simple covariate unconfoundedness:

$D$0

Under this condition, the untreated evolution of the bad control is balanced between treated and untreated units conditional on its pre-treatment value and other exogenous covariates. The ATT is then identified by the standard conditional DiD estimand

$D$1

where $D$2 is the untreated-group mean outcome change conditional on pre-treatment covariates.

This result is notable because the post-treatment bad control does not appear in the estimand. Its omission is valid not because the variable is irrelevant, but because its untreated post-treatment distribution is conditionally balanced and can therefore be integrated out. The paper also gives a non-nested alternative, termed bad-control redundancy, under which the post-treatment untreated covariate adds no information about untreated outcome trends once pre-treatment covariates have been included.

This approach is operationally attractive: researchers can apply existing conditional DiD estimators while including the pre-treatment value of the bad control among the covariates. Its validity, however, depends on a substantive restriction concerning the treatment assignment mechanism for the counterfactual evolution of $D$3.

Covariate unconfoundedness with additional confounders

The second strategy permits variables $D$4 that affect both treatment assignment and the untreated evolution of the bad control. It assumes

$D$5

The corresponding ATT representation is

$D$6

The inner regression estimates untreated outcome trends conditional on the untreated group’s observed bad control. The next conditional expectation integrates this regression over the distribution of the post-treatment bad control among untreated units with the same $D$7 as the treated units. Covariate unconfoundedness justifies using that untreated distribution to reconstruct the counterfactual evolution of $D$8 for treated units.

This nested structure is the paper’s principal conceptual contribution. It separates the problem into two linked counterfactual predictions: first recover how the bad control would have evolved without treatment, then use that recovered distribution to construct the untreated outcome trend required by conditional parallel trends.

Staggered adoption

The paper extends the framework to staggered treatment adoption, with treatment timing $D$9 and group-time effects

$Y_t$0

The multi-period assumptions include no anticipation for both outcomes and covariates, conditional parallel trends across treatment cohorts, covariate unconfoundedness for the untreated covariate process, and overlap with the never-treated group.

The general identification formula conditions on the full untreated covariate path. Although this is theoretically comprehensive, it can be computationally and statistically demanding. Two dimension-reduction results are therefore developed. Under a restriction that outcome changes depend on the covariate history only through its values at the endpoints of the relevant interval, estimation requires conditioning on fewer covariate periods. Under multi-period simple covariate unconfoundedness or bad-control redundancy, the estimator reduces further to conditioning on the covariate value immediately before treatment.

The staggered-adoption results preserve the group-time ATT architecture associated with modern heterogeneous-treatment-effect DiD. Consequently, event-study parameters, overall ATTs, and other aggregations can be constructed from the identified $Y_t$1 values without reverting to a conventional TWFE coefficient.

Estimation and inference

The paper proposes two estimators for the general covariate-unconfoundedness design.

The imputation estimator imposes linear specifications for two nuisance functions: the untreated outcome regression conditional on the bad control and the conditional mean of the bad control among untreated units. The estimated counterfactual bad-control value for treated units is inserted into the untreated outcome regression, producing an imputed untreated trend. The ATT is the treated-group mean outcome change minus the average imputed counterfactual trend.

The second estimator is based on a Neyman-orthogonal score. It combines outcome-regression terms, a propensity score, and an auxiliary conditional odds-ratio regression. The resulting estimator is doubly robust: it is consistent if either the outcome-regression pair is correctly specified or the propensity-score and odds-ratio pair is correctly specified. The construction also supports cross-fitting and nonparametric nuisance estimation.

Under product-rate conditions, each relevant pair of nuisance estimators need only converge sufficiently quickly in $Y_t$2 norm; a common sufficient condition is approximately $Y_t$3 convergence for each nuisance component. The resulting DDML estimator is asymptotically linear and asymptotically normal, with inference based on the estimated influence function. This provides a practical route to flexible estimation in settings where occupation, earnings, and other covariates exhibit nonlinear interactions or high-dimensional heterogeneity.

The methodology also yields pre-tests. Researchers can estimate pseudo-ATTs in pre-treatment periods, where valid identifying assumptions imply no effect. In addition, the treatment effect on the bad control itself can be estimated. Pre-treatment estimates provide diagnostics for the covariate-unconfoundedness structure, whereas post-treatment estimates assess whether the covariate is genuinely affected by treatment.

Application: job displacement and earnings

The empirical application uses a balanced panel of 3,231 NLSY79 respondents observed biennially from 1992 through 2002. Job displacement is defined as involuntary job separation caused by a layoff, job elimination, or workplace closure. The outcome is log earnings, and the bad control is occupation score, defined as the median log hourly wage associated with an occupation using IPUMS data.

Occupation score satisfies the substantive criteria for a bad control. It is plausibly predictive of earnings trajectories, and displacement can induce workers to move into lower-paying occupations. The estimated treatment effect on occupation score is approximately a 2% reduction relative to the counterfactual occupation-score path for displaced workers, with the largest effect occurring around displacement and attenuation thereafter.

Figure 1: Event-study estimates of the effect of job displacement on occupation score, with the largest decline occurring near the displacement period.

The main imputation estimator under covariate unconfoundedness estimates that displacement reduces earnings by approximately 7% on average. The losses are concentrated in the first four years after displacement, with smaller and statistically insignificant effects in later periods.

Figure 2: Imputation event-study estimates showing that earnings losses are concentrated during the first four years following displacement.

The cross-estimator comparison is substantively important. All specifications produce negative effects, generally between 6% and 10%, but the magnitudes differ considerably. TWFE specifications estimate an earnings reduction of nearly 10%, approximately 40% larger in absolute magnitude than the baseline imputation estimate. Imputation specifications that directly include or entirely exclude the bad control produce estimates 3–4% smaller in absolute magnitude than the baseline estimator. The simple-covariate-unconfoundedness ML estimate is approximately 9% larger in magnitude than the baseline, while the doubly robust and general ML estimates are close to the baseline.

Figure 3: Overall ATT estimates across conventional, imputation, doubly robust, and machine-learning estimators relative to the baseline imputation estimate.

These results support two conclusions. First, the qualitative conclusion that displacement substantially reduces earnings is robust. Second, the quantitative magnitude is sensitive to how the occupation variable is handled. In particular, comparing specifications that include and exclude occupation does not constitute a reliable robustness analysis because both specifications may fail for different reasons. The estimates that account explicitly for the counterfactual evolution of occupation are approximately 30% smaller than conventional estimates that directly include or omit the bad control, consistent with the paper’s central identification argument.

Theoretical and practical implications

The theoretical implication is that bad-control problems in DiD are fundamentally counterfactual-distribution problems. It is insufficient to formulate conditional parallel trends using an observed covariate if treatment changes that covariate. Identification requires either a restriction that makes the untreated covariate distribution recoverable from pre-treatment variables or a more general unconfoundedness condition that permits explicit reconstruction of the distribution.

The results also delimit when the proposed methods are applicable. Panel data are essential because the strategies use pre-treatment covariate histories and untreated covariate dynamics. With purely cross-sectional data, the counterfactual evolution of the bad control generally cannot be recovered through these procedures. Repeated cross-sections provide only limited assistance unless additional structure identifies individual-level or distributional transitions.

For applied work, the paper recommends replacing the binary decision to include or exclude a post-treatment covariate with a sequence of diagnostic and modeling decisions:

• determine whether the covariate predicts untreated outcome trends;
• assess whether treatment affects the covariate;
• specify whether pre-treatment covariates suffice to recover its untreated evolution;
• use the nested imputation or doubly robust estimator when additional confounders are required;
• conduct pre-treatment pseudo-effect tests and report treatment effects on the covariate itself;
• avoid interpreting conventional TWFE coefficients as reliable alternatives when treatment effects and covariate responses are heterogeneous.

Several extensions follow naturally. Distributional DiD and change-in-changes methods could recover counterfactual covariate distributions under alternative assumptions, particularly when $Y_t$4 is continuously distributed. Mediation analysis could decompose the total treatment effect into direct and indirect components through the bad control, although that requires assumptions beyond those needed for the overall ATT. More flexible sequential models could accommodate dynamic feedback between outcomes and covariates, especially in long panels. Finally, the accompanying badcontrols R package makes the proposed estimators directly implementable and should facilitate simulation-based sensitivity analyses across alternative assumptions.

Conclusion

“Difference-in-differences with ‘bad controls’” (2608.03881) shows that both including and excluding a treatment-affected, outcome-relevant covariate can invalidate DiD identification. The appropriate response is to recover the covariate’s untreated counterfactual evolution and use that distribution in the conditional parallel-trends argument. The paper supplies a tractable pre-treatment-control strategy, a more general covariate-unconfoundedness framework, staggered-adoption extensions, doubly robust estimators, and DDML inference. Its empirical application demonstrates that the treatment effect on earnings can remain qualitatively stable while changing materially in magnitude depending on how the bad control is handled. The broader implication is that treatment-affected covariates should be incorporated through explicit counterfactual modeling rather than addressed through routine inclusion or deletion.