Bayesian Interrupted Time Series for evaluating policy change on mental well-being: an application to England's welfare reform (2306.15525v1)

Abstract: Factors contributing to social inequalities are also associated with negative mental health outcomes leading to disparities in mental well-being. We propose a Bayesian hierarchical model which can evaluate the impact of policies on population well-being, accounting for spatial/temporal dependencies. Building on an interrupted time series framework, our approach can evaluate how different profiles of individuals are affected in different ways, whilst accounting for their uncertainty. We apply the framework to assess the impact of the United Kingdoms welfare reform, which took place throughout the 2010s, on mental well-being using data from the UK Household Longitudinal Study. The additional depth of knowledge is essential for effective evaluation of current policy and implementation of future policy.

• The paper develops a Bayesian hierarchical Interrupted Time Series framework to rigorously evaluate policy changes' impact on mental well-being, accounting for space and time.
• Applied to England's welfare reform, the model reveals a substantial national increase in psychological distress among unemployed individuals exposed to Universal Credit.
• Findings show significant geographic and socioeconomic disparities in the welfare reform's impact, indicating the model's value in revealing nuanced effects and its potential for use in other policy evaluations.

The paper, titled "Bayesian Interrupted Time Series for evaluating policy change on mental well-being: an application to England's welfare reform," presents a methodological framework for evaluating the impact of policy changes on mental well-being using a Bayesian hierarchical model. This framework is particularly applied to assess the impact of the United Kingdom's welfare reform, specifically Universal Credit (UC), on mental well-being by leveraging data from the UK Household Longitudinal Study (UKHLS).

Key Contributions:

1. Bayesian Hierarchical Model: This paper introduces a Bayesian hierarchical model that accounts for both spatial and temporal dependencies. The model is designed within an Interrupted Time Series (ITS) framework, allowing for the evaluation of policy impacts on different population profiles while accounting for uncertainties.
2. Application to Welfare Reform: The paper examines the causal effects of the UK's UC welfare reform on mental health outcomes. The paper meticulously models individual psychological distress (measured via the General Health Questionnaire-12) in association with UC exposure defined by employment status.
3. Contextual Awareness: The paper introduces the concept of 'contextual awareness' to UC, defining it at the local authority level as a measure of awareness of UC rollout. This variable accounts for those indirectly influenced by UC despite not participating themselves.
4. Sensitive Analysis and Inequalities: It provides a detailed sensitivity analysis on the definition of contextual awareness and the temporal initiation of the intervention. This feature allows exploration of inequalities related to socioeconomic status, ethnic background, and geographic location.

Methodology:

• Participant Selection: The analysis focuses on working-age individuals (16-64 years) residing in England, considering various demographic variables and excluding those with permanent disabilities.
• Confounding Variables: The model controls for individual-level confounders, such as age, education, ethnicity, marital status, and sex, alongside area-level confounders like social deprivation and ethnic composition.
• Statistical Model: The model uses a Bernoulli distribution to represent psychological distress and incorporates random effects for time and space to address unstructured variation. The central parameters are estimated through Integrated Nested Laplace Approximation (INLA), offering computationally efficient Bayesian inference.
• Standardised Percentage Change: This metric quantifies the intervention's effect on psychological distress, considering broader population trends.

Results:

• Temporal Profile Analysis: The national prevalence of psychological distress is found to be significantly higher in the exposed (unemployed) populations compared to the control populations. A peak immediately before the intervention suggests anticipatory stress related to policy change.
• Spatial Variation: Different Lower Tier Local Authorities (LTLAs) showed variability in the change in psychological distress, emphasizing the importance of geographic considerations in policy impact analysis.
• Socioeconomic Disparities: Analyses reveal significant variation in impact based on socioeconomic and demographic profiles, with some subgroups experiencing increased stress due to UC while others show resilience.
• Overall Impact: Nationally, the intervention led to a substantial increase (15.30%) in psychological distress among the unemployed, though this effect varied regionally.

Conclusion:

The presented Bayesian hierarchical framework effectively addresses the complexities of evaluating policy impact on mental well-being. By capturing spatial and temporal dimensions, it aids in understanding nuanced influences at the subnational level and provides critical insights into unintended policy consequences. This model is virtually adaptable to other policy contexts or health outcomes, extending its utility beyond the specific case of UC.