The "Colonial Impulse" of Natural Language Processing: An Audit of Bengali Sentiment Analysis Tools and Their Identity-based Biases (2401.10535v1)

Abstract: While colonization has sociohistorically impacted people's identities across various dimensions, those colonial values and biases continue to be perpetuated by sociotechnical systems. One category of sociotechnical systems--sentiment analysis tools--can also perpetuate colonial values and bias, yet less attention has been paid to how such tools may be complicit in perpetuating coloniality, although they are often used to guide various practices (e.g., content moderation). In this paper, we explore potential bias in sentiment analysis tools in the context of Bengali communities that have experienced and continue to experience the impacts of colonialism. Drawing on identity categories most impacted by colonialism amongst local Bengali communities, we focused our analytic attention on gender, religion, and nationality. We conducted an algorithmic audit of all sentiment analysis tools for Bengali, available on the Python package index (PyPI) and GitHub. Despite similar semantic content and structure, our analyses showed that in addition to inconsistencies in output from different tools, Bengali sentiment analysis tools exhibit bias between different identity categories and respond differently to different ways of identity expression. Connecting our findings with colonially shaped sociocultural structures of Bengali communities, we discuss the implications of downstream bias of sentiment analysis tools.

• The paper examines Bengali sentiment tools to audit bias rooted in colonial legacies against gender, religion, and nationality.
• It applies an algorithmic audit using the BIBED dataset to measure identity biases in outputs from popular NLP tools.
• Findings call for intersectional collaboration and more inclusive design to mitigate social biases in language technology.

Introduction

Natural language processing (NLP) has burgeoned with advancements that help machines interpret human emotions conveyed through text via sentiment analysis. While essential, this NLP domain's reliance on quantifying complex emotional experiences can miss nuances, inadvertently enforcing social and technical biases. Particularly in non-English NLP work, there is a disparity in research, disadvantaging languages like Bengali. This paper takes strides to analyze the biases in Bengali sentiment analysis (BSA) tools, centered on identity categories profoundly affected by colonialism.

Literature Review

The critical discourse on NLP has underscored an imbalance in linguistic research focus and resources, which is starkly highlighted in the comparison between English and Bengali language tools. Drawing from the concept of sociotechnical systems, the paper advances the notion that sentiment analysis tools interweave with social interaction—affected by the developers and the usage context. Embedded biases in these tools can replicate colonial ideologies and identity categorizations. Previous work outlines that while identities are often multidimensional, colonial impressions have historically altered self-perception concerning gender, religion, and nationality in Bengali societies. This paper further interrogates how sentiment tools process these identities.

Methods

The paper performs an algorithmic audit on sentiment analysis tools sourced from the Python Package Index (PyPI) and GitHub. Identity expressions examined are gender, religion, and nationality, drawing from the Bengali community's historical interactions with colonization. An existing Bengali Identity Bias Evaluation Dataset (BIBED) served as the benchmark to evaluate if BSA tools consistently discriminate against particular identities. By interacting with these tools and evaluating the outputs, the paper quantifies bias and its relationship with the developer demographics.

Results and Discussion

The findings are telling. Despite tools being fed sentences with identical context, varying responses are noted across different BSA tools, debunking the claim of universality that often upholds sentiment analysis methodologies. Significant biases were detected toward specific identities—gender, religion, and nationality. Contrary findings were that while there is bias in BSA tools, there isn't a clear link to the developers' demographic backgrounds.

The paper's discussion brings to light the "colonial impulse" in sentiment analysis tools, reflecting colonial power dynamics by favoring certain identities over others. It calls for intersectional collaboration among developers to ensure that design processes are inclusive and account for bias in sentiment analysis. Furthermore, the repercussions of its findings are pivotal when considering downstream applications like content moderation, where biases could amplify social divisions and hamper inclusive user engagement.

Conclusion

In conclusion, the paper provides a critical examination of BSA tools, showcasing the persistence of colonial values in technology. Emphasizing the need for diversity and intersectionality in technology development, the work calls for an engineering activism that is cognizant not just of technical prowess but also of the social fabric that it is inevitably woven into. This resonates deeply within the CHI community, situating the paper at the confluence of technology, critical theory, and social justice.