Assessing Social and Intersectional Biases in Contextualized Word Representations (1911.01485v1)

Abstract: Social bias in machine learning has drawn significant attention, with work ranging from demonstrations of bias in a multitude of applications, curating definitions of fairness for different contexts, to developing algorithms to mitigate bias. In natural language processing, gender bias has been shown to exist in context-free word embeddings. Recently, contextual word representations have outperformed word embeddings in several downstream NLP tasks. These word representations are conditioned on their context within a sentence, and can also be used to encode the entire sentence. In this paper, we analyze the extent to which state-of-the-art models for contextual word representations, such as BERT and GPT-2, encode biases with respect to gender, race, and intersectional identities. Towards this, we propose assessing bias at the contextual word level. This novel approach captures the contextual effects of bias missing in context-free word embeddings, yet avoids confounding effects that underestimate bias at the sentence encoding level. We demonstrate evidence of bias at the corpus level, find varying evidence of bias in embedding association tests, show in particular that racial bias is strongly encoded in contextual word models, and observe that bias effects for intersectional minorities are exacerbated beyond their constituent minority identities. Further, evaluating bias effects at the contextual word level captures biases that are not captured at the sentence level, confirming the need for our novel approach.

An Analysis of Social and Intersectional Biases in Contextualized Word Representations

The paper "Assessing Social and Intersectional Biases in Contextualized Word Representations" by Yi Chern Tan and L. Elisa Celis addresses the complex issue of social biases within NLP, focusing on contextual word representations. The paper builds upon existing research that identified biases in traditional word embeddings and extends this to contextual models like BERT and GPT-2, which have become critical in advancing NLP capabilities.

Key Insights

The paper conducts a comprehensive investigation into how state-of-the-art contextual word models encode biases related to gender, race, and intersectional identities. The authors introduce a novel methodology that evaluates bias at the contextual word level, which captures nuances that are often overlooked when analyzing just sentence-level representations. This approach remains distinct from context-free embeddings and demonstrates its effectiveness in showcasing the propagation of social biases.

Methodology

The central methodological contribution of this paper is the adaptation of the Word Embedding Association Tests (WEATs) into Sentence Encoder Association Tests (SEATs), further extending them to what the authors call "contextualized word representations." This involves leveraging embedding association tests to measure biases by examining the association between concepts (e.g., gendered names) and attributes (e.g., occupations) at the contextual word level. The methodology aims to uncover biases in models like BERT and GPT-2, providing evidence of significant gender and racial biases encoded into these contextualized representations.

Numerical Results

The empirical analysis revealed that biases in language corpora—such as a greater occurrence of male pronouns compared to female pronouns—transmit to the models trained on these corpora. Detailed quantitative results display that racial bias is more prevalent than gender bias in these models. This critical finding emphasizes the urgent need for bias mitigation in LLMs to ensure fair and unbiased NLP applications.

Implications

The acknowledgment of encoded biases in prevalent NLP models has far-reaching implications. On a practical level, it suggests necessary revisions in the training datasets and models used in sensitive applications, including content recommendations, sentiment analysis, and automated hiring systems. The theoretical impact highlights the intrinsic biases perpetuated by the data, suggesting that even as models gain sophistication, their fairness must be critically evaluated.

Future Directions

The paper opens pathways for future research, especially in developing de-biasing techniques specifically for contextual word models. There is a call for exploring whether the bias varies across transformer layers and model sizes. The authors also encourage the documentation of datasets used in training these models, akin to nutrition labels, to inform users of potential biases embedded within NLP systems.

The authors' comprehensive examination of social and intersectional biases in contextual word representations sets a precedent for rigorous evaluation and mitigation of biases in NLP. As LLMs become omnipresent, such meticulous analyses and methodologies not only advance NLP research but also ensure its equitable application across diverse populations.