Detecting Emergent Intersectional Biases: Contextualized Word Embeddings Contain a Distribution of Human-like Biases (2006.03955v5)

Abstract: With the starting point that implicit human biases are reflected in the statistical regularities of language, it is possible to measure biases in English static word embeddings. State-of-the-art neural LLMs generate dynamic word embeddings dependent on the context in which the word appears. Current methods measure pre-defined social and intersectional biases that appear in particular contexts defined by sentence templates. Dispensing with templates, we introduce the Contextualized Embedding Association Test (CEAT), that can summarize the magnitude of overall bias in neural LLMs by incorporating a random-effects model. Experiments on social and intersectional biases show that CEAT finds evidence of all tested biases and provides comprehensive information on the variance of effect magnitudes of the same bias in different contexts. All the models trained on English corpora that we study contain biased representations. Furthermore, we develop two methods, Intersectional Bias Detection (IBD) and Emergent Intersectional Bias Detection (EIBD), to automatically identify the intersectional biases and emergent intersectional biases from static word embeddings in addition to measuring them in contextualized word embeddings. We present the first algorithmic bias detection findings on how intersectional group members are strongly associated with unique emergent biases that do not overlap with the biases of their constituent minority identities. IBD and EIBD achieve high accuracy when detecting the intersectional and emergent biases of African American females and Mexican American females. Our results indicate that biases at the intersection of race and gender associated with members of multiple minority groups, such as African American females and Mexican American females, have the highest magnitude across all neural LLMs.

Detecting Emergent Intersectional Biases in Contextualized Word Embeddings

The paper "Detecting Emergent Intersectional Biases: Contextualized Word Embeddings Contain a Distribution of Human-like Biases" by Wei Guo and Aylin Caliskan introduces novel methodologies for identifying and quantifying biases in state-of-the-art neural LLMs, focusing on both social and intersectional biases in contextualized word embeddings (CWEs). The work addresses the need for sophisticated approaches to unveil biases embedded in language representations that can impact downstream NLP tasks and applications.

Methodological Innovations

The paper proposes several methods to detect and measure biases in neural LLMs:

1. Contextualized Embedding Association Test (CEAT): This method extends the traditional Word Embedding Association Test (WEAT) to CWEs by sampling a large number of embedding variations to simulate a distribution of bias effects. CEAT uses a random-effects model to characterize the magnitude and variability of biases in CWEs, providing a robust statistical summary of bias within LLMs.
2. Intersectional Bias Detection (IBD): IBD autonomously identifies attributes highly associated with intersectional group members, such as African American females or Mexican American females, by analyzing static word embeddings. The method demonstrates high accuracy (81.6% and 82.7% for two tested groups), far exceeding random identification rates (14.3% and 13.3%).
3. Emergent Intersectional Bias Detection (EIBD): This method targets "emergent" biases unique to certain intersectional groups, capturing associations that do not overlap with biases of constituent minority identities. EIBD also shows significant accuracy, particularly in identifying biases relevant to African American and Mexican American females, compared to random chances.

Empirical Findings

The empirical evaluation confirms that all tested neural LLMs contained biases, with varying magnitudes dependent on the level of contextualization. GPT-2 exhibited the smallest overall bias, which suggests that more contextualized CWEs contain lower bias magnitudes. Notably, the models consistently showed stronger associations with intersectional group members, highlighting an area requiring targeted bias mitigation efforts.

The paper emphasizes the significance of understanding biases related to intersectional identities, which conventional single-category focus approaches might overlook, thus leading to incomplete bias measurements. This work contributes by filling this gap and offering methodologies that uncover complex stereotype structures within LLMs.

Implications and Future Work

This research has substantial implications for the design and deployment of NLP systems:

• Bias Mitigation: The findings and methodologies serve as a foundation for developing strategies to mitigate biases in LLMs, potentially improving fairness in automated systems deployed across socially impactful sectors like hiring and healthcare.
• Intersectional Dimensions: Addressing intersectional biases is critical for enhancing the inclusiveness and fairness of AI systems, ensuring that multiple minority identities are adequately considered when assessing model biases.
• LLM Development: Future advancements in AI could focus on refining LLMs to learn more contextually rich and bias-free representations, aided by tools like CEAT, IBD, and EIBD.

Summarily, this paper demonstrates that sophisticated methodologies are necessary to holistically assess biases within neural LLMs, particularly understanding the nuanced biases affecting intersectional groups, which are critical for progressing towards equitable and trustworthy AI systems.