(Unfair) Norms in Fairness Research: A Meta-Analysis (2407.16895v1)

Abstract: Algorithmic fairness has emerged as a critical concern in AI research. However, the development of fair AI systems is not an objective process. Fairness is an inherently subjective concept, shaped by the values, experiences, and identities of those involved in research and development. To better understand the norms and values embedded in current fairness research, we conduct a meta-analysis of algorithmic fairness papers from two leading conferences on AI fairness and ethics, AIES and FAccT, covering a final sample of 139 papers over the period from 2018 to 2022. Our investigation reveals two concerning trends: first, a US-centric perspective dominates throughout fairness research; and second, fairness studies exhibit a widespread reliance on binary codifications of human identity (e.g., "Black/White", "male/female"). These findings highlight how current research often overlooks the complexities of identity and lived experiences, ultimately failing to represent diverse global contexts when defining algorithmic bias and fairness. We discuss the limitations of these research design choices and offer recommendations for fostering more inclusive and representative approaches to fairness in AI systems, urging a paradigm shift that embraces nuanced, global understandings of human identity and values.

A Reflexive Critique of Algorithmic Fairness Research

The paper "(Unfair) Norms in Fairness Research: A Meta-Analysis" by Jennifer Chien et al. presents a comprehensive meta-analysis of algorithmic fairness research published in leading conferences, specifically AIES and FAccT from 2018 to 2022. This effort critically examines the embedded norms and biases within the field, uncovering underlying issues that shape and potentially skew the development of fair AI systems. The analysis highlights two primary concerns: a US-centric bias in research and a widespread reliance on binary codifications of human identity.

Research Context and Methodology

Chien et al. undertook an extensive meta-analysis, examining 139 papers from AIES and FAccT conferences. The paper scrutinized a range of variables including dataset characteristics, author affiliations, sensitive attributes examined, and the formulation and methodological approaches of fairness studies. The authors incorporated a reflexive stance, emphasizing how the identity and background of researchers might shape their work.

Key Findings

US-Centric Bias

A striking pattern identified in the analysis is a predominant US-centric perspective. Most research papers in the sample were authored by researchers affiliated with US institutions, and the datasets employed often originated from the US as well. Key findings supporting this include:

• Authorship: US-based authors represented a staggering 80.6% of the sample.
• Dataset Provenance: 72.6% of datasets analyzed in the fairness studies were US-sourced.

This bias is not only in geographic representation but also in the framing of issues. Sensitive attributes, particularly race, were predominantly conceptualized within a Black/White binary framework. This racial dichotomy reflects American societal norms and legal definitions but oversimplifies the complex spectrum of racial identities globally.

Binary Codifications

The second major issue identified is the common use of binary divisions for sensitive attributes such as gender, race, and age. For example:

• Gender: Frequently reduced to "male/female".
• Race: Often categorized as "Black/White" or "Black/non-Black".
• Age: Typically segmented into two broad groups, such as "young/old".

These binary categorizations overlook the multi-dimensionality and fluidity of human identities. The paper notes that most studies did not provide precise definitions for these categories, further compounding the problem. This approach can obscure intersectional issues, ignoring how various aspects of identity interact and affect individuals’ experiences with AI systems.

Implications

The implications of these findings are multifaceted, impacting both theoretical and practical aspects of algorithmic fairness research:

1. Theoretical Frameworks: The paper calls for a reassessment of normativity within fairness research. The predominance of US-centric perspectives and simplistic, binary categorizations necessitates a broadened viewpoint that includes diverse global perspectives and more nuanced understandings of identity.
2. Practical Applications: Algorithmic systems trained on these biased datasets and definitions risk perpetuating existing inequalities when deployed. The paper advocates for reflexivity and transparency in the research process. It also promotes the inclusion of marginalized voices both in dataset construction and in the research teams themselves.
3. Policy and Regulation: Policymakers and regulators should consider the broader scope of fairness beyond the currently dominant paradigms. Policies should encourage and perhaps require the use of diverse datasets and intersectional approaches in fairness assessments.

Future Directions

Looking ahead, the paper suggests several pathways to mitigate these issues:

• Diversification of Data and Authorship: Broader, global representation in dataset origins and author affiliations could mitigate US-centrism.
• Nuanced Definitions: Adoption of more comprehensive, non-binary categorizations for sensitive attributes can improve the representativeness and accuracy of fairness studies.
• Participatory Approaches: Involving diverse communities in the research process can enhance the relevance and fairness of AI systems.

Conclusion

Chien et al.'s meta-analysis underscores the critical need for the fairness research community to scrutinize and evolve its established norms and practices. By recognizing and addressing US-centric biases and overly simplistic binary categorizations of identity, the field can move towards more inclusive and genuine representations of global human experiences. This reflexive approach is vital to developing AI systems that are truly fair and equitable for all users.