Reducing Population-level Inequality Can Improve Demographic Group Fairness: a Twitter Case Study

Abstract: Many existing fairness metrics measure group-wise demographic disparities in system behavior or model performance. Calculating these metrics requires access to demographic information, which, in industrial settings, is often unavailable. By contrast, economic inequality metrics, such as the Gini coefficient, require no demographic data to measure. However, reductions in economic inequality do not necessarily correspond to reductions in demographic disparities. In this paper, we empirically explore the relationship between demographic-free inequality metrics -- such as the Gini coefficient -- and standard demographic bias metrics that measure group-wise model performance disparities specifically in the case of engagement inequality on Twitter. We analyze tweets from 174K users over the duration of 2021 and find that demographic-free impression inequality metrics are positively correlated with gender, race, and age disparities in the average case, and weakly (but still positively) correlated with demographic bias in the worst case. We therefore recommend inequality metrics as a potentially useful proxy measure of average group-wise disparities, especially in cases where such disparities cannot be measured directly. Based on these results, we believe they can be used as part of broader efforts to improve fairness between demographic groups in scenarios like content recommendation on social media.

• The paper establishes that demographic-free measures like the Gini coefficient and T1PS are strongly correlated with demographic bias metrics.
• It employs a linked Twitter-voter dataset to compute daily engagement statistics and applies Spearman’s rank correlation for both marginal and intersectional analyses.
• The findings imply that reducing overall engagement inequality can serve as a proxy for improving fairness when demographic data is unavailable.

Reducing Population-Level Inequality Can Improve Demographic Group Fairness: A Twitter Case Study

The paper, "Reducing Population-level Inequality Can Improve Demographic Group Fairness: a Twitter Case Study," explores the intersection of algorithmic fairness and social media interactions. Utilizing a comprehensive dataset of Twitter users and their demographic information, the authors explore how demographic-free economic inequality metrics such as the Gini coefficient and Top 1\% Share (T1PS) correlate with demographic bias metrics.

Key Investigation and Methodology

The central question the paper seeks to answer is whether demographic-free inequality measures can serve as proxies for demographic disparities when demographic information is unavailable. This inquiry is particularly crucial in industrial applications where collecting demographic data may be infeasible or illegal. The research follows these steps:

1. Dataset Construction: The study utilizes a dataset of Twitter users linked to voter records, providing demographic labels such as age, gender, race, and political affiliation.
2. Metric Calculation: Daily inequality metrics (Gini and T1PS) are computed based on user engagements (likes and retweets), alongside demographic bias metrics (MAD and IMM) for various demographic groups.
3. Correlation Analysis: Spearman's rank correlation coefficient measures the relationship between demographic-free inequality metrics and demographic disparity metrics, both for marginal (single attributes) and intersectional (pairs of attributes) analyses.

Findings and Implications

The study reveals a notable positive correlation between inequality metrics and demographic bias metrics, particularly for marginal attributes. For instance, the Gini coefficient and T1PS exhibit correlations as high as 0.61 with MAD, suggesting that reducing overall inequality could potentially alleviate demographic disparities. However, a notable exception is the low correlation with political view, which underscores the nuanced nature of these relationships. In intersectional analyses, correlations are even stronger, reaching up to 0.78, indicating that demographic-free metrics might effectively capture complex multi-attribute disparities.

These findings have significant implications:

• Practical Utility: In scenarios where demographic data is sparse or unavailable, inequality metrics can guide system adjustments to promote fairness.
• Platform Policies: For platforms like Twitter, this approach could inform content recommendation systems, potentially reducing demographic biases in user engagement without direct demographic data.

Theoretical Insights

From a theoretical standpoint, the paper contributes to algorithmic fairness literature by empirically validating the use of demographic-free metrics. It illustrates contexts where these metrics can reflect underlying demographic disparities, thereby supporting their use in fairness-enhancing interventions.

Future Directions in AI

Several future research avenues are suggested:

1. Engagement vs. Impression Data: Future studies should explore the use of impression data to provide a more direct measure of content distribution.
2. Global Scope and Other Platforms: Extending this analysis to a global user base and other social media platforms would validate the findings' generalizability.
3. Causal Experiments: Implementing A/B testing could establish causal relationships between inequality reduction and demographic fairness.
4. Granular Analyses: Further studies might disaggregate influencer effects from natural inequalities to refine conclusions about fairness interventions.

In summary, the paper advances our understanding of how demographic-free inequality metrics can potentially serve as proxies for measuring and addressing demographic disparities in large sociotechnical systems. It presents a foundational step towards more equitable algorithmic practices, especially in environments where demographic data collection is challenging.