Dice Question Streamline Icon: https://streamlinehq.com

Assess directional bias from national-level aggregation of heterogeneous policies

Ascertain how to assess the directional bias introduced by using national-level averages of heterogeneous subnational policy implementations as predictors in time-series regressions of epidemic outcomes, taking into account potential biases such as those implied by Jensen's Inequality.

Information Square Streamline Icon: https://streamlinehq.com

Background

The paper critiques the use of national-level policy intensity measures that aggregate heterogeneous subnational implementations. Given the nonlinear dynamics of epidemics and substantial within-country variability in both outcomes and policy intensity, averaging can distort the relationship between policy and outcomes.

The author explicitly states that it is not clear how to assess the directional bias induced by such aggregation, highlighting an unresolved methodological challenge in evaluating policy effects from aggregated data.

References

It is not clear exactly how to assess the directional bias that may induced by its use, but even if there is not such bias in direction, dependent and independent variables measured with noise tend to bias estimated coefficients towards the null as its harder to detect signals in such situations - and as the authors report, they do observe that most of their policy coefficients are relatively small and most are not significant.

— Analysis of Potential Biases and Validity of Studies Using Multiverse Approaches to Assess the Impacts of Government Responses to Epidemics (2409.06930 - Goldhaber-Fiebert, 11 Sep 2024) in Approach and Findings – Bias Due to Level of Aggregation?