Fair is Better than Sensational:Man is to Doctor as Woman is to Doctor (1905.09866v2)

Abstract: Analogies such as "man is to king as woman is to X" are often used to illustrate the amazing power of word embeddings. Concurrently, they have also been used to expose how strongly human biases are encoded in vector spaces built on natural language, like "man is to computer programmer as woman is to homemaker". Recent work has shown that analogies are in fact not such a diagnostic for bias, and other methods have been proven to be more apt to the task. However, beside the intrinsic problems with the analogy task as a bias detection tool, in this paper we show that a series of issues related to how analogies have been implemented and used might have yielded a distorted picture of bias in word embeddings. Human biases are present in word embeddings and need to be addressed. Analogies, though, are probably not the right tool to do so. Also, the way they have been most often used has exacerbated some possibly non-existing biases and perhaps hid others. Because they are still widely popular, and some of them have become classics within and outside the NLP community, we deem it important to provide a series of clarifications that should put well-known, and potentially new cases into the right perspective.

Analyzing Bias in Word Embeddings: Limitations of Analogies

The paper "Fair is Better than Sensational: Man is to Doctor as Woman is to Doctor" by Malvina Nissim, Rik van Noord, and Rob van der Goot critically examines the use of analogy tasks in assessing biases embedded within word embeddings. The authors argue that while analogies such as "man is to king as woman is to queen" have been showcased to demonstrate the capacity of word embeddings, they are less effective for diagnosing bias, offering clarification on the misconceptions surrounding this method.

Criticism of Analogy Tasks

The paper highlights a significant flaw in using analogies to detect bias, stemming from the structural limitations and subjective implementation biases that may result in distorted outcomes. Analogies, structured as $A:B::C:D$, demand distinct terms, a requirement not always met in example scenarios. For instance, queries like "man is to doctor as woman is to nurse" illustrate societal stereotypes but are constrained by implementations that prevent returning $D$ equal to any of the input terms, potentially leading to skewed interpretations of bias.

Methodological Insights

To address these concerns, the authors analyzed various analogy detection strategies, including the original 3cosadd method, the revised 3cosmul method, and a formula proposed by Bolukbasi et al., 2016. Each method carries intrinsic biases and subjective choices that influence the returned results, impacting the integrity of bias detection outcomes. Notably, unconstrained versions of these algorithms often yield different, sometimes less biased results, emphasizing the consequences of methodological choices in perceived bias.

Subjective Influence in Query Construction

The paper underscores the degree to which human biases infiltrate the formation and expectations of analogy queries. Researchers often choose queries expecting biased answers, consequently influencing the detection and reporting of bias. Additionally, the selective reporting of biased terms from deeper in the results list, rather than consistently reporting the top terms, further clouds the evaluation of bias within embeddings.

Implications and Recommendations

By dissecting the limitations of analogy tasks, the authors call for greater transparency in bias detection methodologies, stressing the significance of methodological scrutiny in addressing bias within embeddings. The paper also urges the research community to move towards more reliable methods of bias assessment in word embeddings, advocating for continued awareness of how underlying societal biases manifest computationally.

Future Directions

While the limitations of analogy tasks are evident, the exploration of biases in embeddings remains critical to the advancement of fair AI systems. Future research may focus on harnessing alternative bias detection methods that mitigate the subjective biases present in current approaches. The quest for transparency and objectivity in bias assessment will likely drive innovation and refinement in the paper of word embeddings, contributing to more equitable applications of AI technologies.

This paper marks a significant contribution to the discourse on bias in AI, challenging the field to reassess traditional analogical frameworks and pursue more insightful methodologies for embedding bias detection. As researchers dive deeper into uncovering biases inherent in computational models, the teachings from this paper will serve as a benchmark for ensuring rigour and fairness in AI research and applications.