Simpson's paradox in Covid-19 case fatality rates: a mediation analysis of age-related causal effects

Abstract: We point out an instantiation of Simpson's paradox in Covid-19 case fatality rates (CFRs): comparing a large-scale study from China (17 Feb) with early reports from Italy (9 Mar), we find that CFRs are lower in Italy for every age group, but higher overall. This phenomenon is explained by a stark difference in case demographic between the two countries. Using this as a motivating example, we introduce basic concepts from mediation analysis and show how these can be used to quantify different direct and indirect effects when assuming a coarse-grained causal graph involving country, age, and case fatality. We curate an age-stratified CFR dataset with >750k cases and conduct a case study, investigating total, direct, and indirect (age-mediated) causal effects between different countries and at different points in time. This allows us to separate age-related effects from others unrelated to age and facilitates a more transparent comparison of CFRs across countries at different stages of the Covid-19 pandemic. Using longitudinal data from Italy, we discover a sign reversal of the direct causal effect in mid-March which temporally aligns with the reported collapse of the healthcare system in parts of the country. Moreover, we find that direct and indirect effects across 132 pairs of countries are only weakly correlated, suggesting that a country's policy and case demographic may be largely unrelated. We point out limitations and extensions for future work, and, finally, discuss the role of causal reasoning in the broader context of using AI to combat the Covid-19 pandemic.

• The paper presents a mediation analysis framework that distinguishes between total, direct, and indirect age-related effects on Covid-19 fatality rates.
• The paper reveals that despite lower fatality rates within age groups in Italy, the overall rate is higher due to an older case distribution, illustrating Simpson's paradox.
• The paper identifies temporal shifts—such as mid-March reversals in direct effects—that highlight the impact of healthcare overload and demographic variability on fatality metrics.

Mediation Analysis of Age-Related Effects in Covid-19 Case Fatality Rates: Understanding Simpson's Paradox

The paper "Simpson's Paradox in Covid-19 Case Fatality Rates: A Mediation Analysis of Age-Related Causal Effects" explores an instance of Simpson's paradox occurring in the comparison of Covid-19 case fatality rates (CFRs) between China and Italy. The paradox is exemplified by the observation that, while Italian CFRs are lower than those of China across all age groups, the overall CFR is higher in Italy. This paradoxical situation is attributed to the stark differences in age demographics of confirmed cases in each country.

Theoretical Framework and Methodology

The authors employ mediation analysis to dissect the underlying causal mechanisms driving this apparent paradox. They introduce a simple causal graph model involving three variables: the reporting country (C), the patient's age group (A), and the medical outcome (F). This model posits that the country's influence on fatality is mediated by the age group, allowing for the separation of direct and indirect age-related effects on CFRs.

Causal inference methods are applied to quantify these distinct effects:

1. Total Causal Effect (TCE): Captures the overall impact of changing countries on CFR.
2. Controlled Direct Effect (CDE): Measures direct country effect on CFR within a fixed age group.
3. Natural Direct Effect (NDE): Assesses direct country effect, controlling for age distribution.
4. Natural Indirect Effect (NIE): Evaluates indirect effects mediated by age.

The study leverages longitudinal CFR data from Italy and age-stratified data from multiple countries, facilitating a nuanced examination of the temporal evolution and geographical differences in these causal effects.

Key Findings

Through their mediation analysis, the authors reveal several critical insights:

• The direct effect (NDE) exhibited a temporal sign reversal in mid-March 2020 in Italy, correlating with reports of healthcare system overload. Initially, Italian approaches yielded lower CFRs when controlling for age, but as conditions worsened, the direct effect became detrimental.
• Across multiple countries, direct and indirect effects are weakly correlated, indicating that a country’s pandemic response and its demographic case profile are largely independent.
• The study underscores significant variations in CFRs due to age demographics, with countries like South Africa, Argentina, and Colombia benefiting from a younger case demographic.

Implications and Future Directions

This paper cautions against simplistic comparisons of overall CFRs across countries without considering demographic distributions and highlights the utility of causal mediation analysis in revealing underlying contributors to observed fatality rates. It also stresses the need for more granular data and sophisticated causal models to better understand the diverse factors influencing Covid-19 fatality beyond age, such as healthcare infrastructure and policy differences.

Moreover, the study suggests opportunities for future research to extend the causal model to incorporate additional mediators or confounders, such as socio-economic factors or testing strategies, which could provide further insights into pandemic outcomes and inform policy interventions.

This work is a pertinent reminder of the complexities inherent in epidemiological data interpretation and the vital role played by causal reasoning in the application of artificial intelligence and data science to real-world problems. As we advance in the ability to collect and analyze vast datasets, enhancing our causal understanding will be crucial in combating pandemics effectively and equitably.