The Effect of Human v/s Synthetic Test Data and Round-tripping on Assessment of Sentiment Analysis Systems for Bias (2401.12985v1)

Published 15 Jan 2024 in cs.CL

Abstract: Sentiment Analysis Systems (SASs) are data-driven AI systems that output polarity and emotional intensity when given a piece of text as input. Like other AIs, SASs are also known to have unstable behavior when subjected to changes in data which can make it problematic to trust out of concerns like bias when AI works with humans and data has protected attributes like gender, race, and age. Recently, an approach was introduced to assess SASs in a blackbox setting without training data or code, and rating them for bias using synthetic English data. We augment it by introducing two human-generated chatbot datasets and also consider a round-trip setting of translating the data from one language to the same through an intermediate language. We find that these settings show SASs performance in a more realistic light. Specifically, we find that rating SASs on the chatbot data showed more bias compared to the synthetic data, and round-tripping using Spanish and Danish as intermediate languages reduces the bias (up to 68% reduction) in human-generated data while, in synthetic data, it takes a surprising turn by increasing the bias! Our findings will help researchers and practitioners refine their SAS testing strategies and foster trust as SASs are considered part of more mission-critical applications for global use.

References (19)

Citations (1)

View on Semantic Scholar

Summary

The paper demonstrates that using human-annotated test data and round-trip translation reveals significant differences in bias assessments compared to synthetic data.
It employs causal models and diverse datasets, including chatbot interactions and synthetic sources, to quantify bias through metrics like Weighted Rejection Scores and Deconfounding Impact Estimations.
The study shows that round-trip translation reduces bias in human data while increasing bias in synthetic data, underscoring the need for real-world evaluation in sentiment analysis.

The Effect of Human vs. Synthetic Test Data and Round-tripping on Assessment of Sentiment Analysis Systems for Bias

Introduction

The paper "The Effect of Human v/s Synthetic Test Data and Round-tripping on Assessment of Sentiment Analysis Systems for Bias" addresses the challenge of assessing bias in Sentiment Analysis Systems (SASs). Sentiment Analysis Systems output polarity and emotional intensity based on input text, but they may exhibit bias, particularly when dealing with protected attributes such as gender and race. This paper extends previous research by introducing two human-annotated datasets and examining the impact of round-trip translation on bias assessments.

Background

Bias in AI systems, including SASs, is a critical barrier to their adoption in trust-critical areas such as healthcare and education. Previous studies have primarily focused on synthetic English datasets, which may not fully capture bias in real-world scenarios. Furthermore, cross-lingual settings, where data is translated between languages, can also affect bias in SASs.

The authors investigate the effects of human-generated data and round-trip translation in bias assessment of SASs, using datasets like the Equity Evaluation Corpus (EEC) for controlled experiments.

Causal Model for Bias Assessment

The paper utilizes causal models to assess bias in SASs. A causal diagram represents cause-effect relationships among the attributes in a system. The model considers desirable attributes, protected attributes, and confounders that could skew sentiment analysis results.

Figure 1: Causal model for rating SASs.

Data and Methodology

The research incorporates three datasets: a synthetic dataset (SD) and two human-generated datasets (HD1 and HD2). The synthetic dataset is derived from the EEC, while the human datasets come from chatbot interactions (ALLURE and Unibot data). For each dataset, sentiment values are compared across different gender and race classes to assess statistical and confounding bias.

Figure 2: Snapshot of the preprocessed ALLURE dataset (HD1).

Figure 3: Snapshot of the preprocessed Unibot data (HD2).

Experiments and Analysis

Impact of Round-trip Translation

The authors explore how round-trip translation, where text is translated from one language back to the original through an intermediate language, affects bias in SASs. The paper uses Danish and Spanish as intermediate languages, assessing any changes in the statistical and confounding bias of SASs.

Figure 4: Methodology for comparing bias scores on original and round-trip translated data.

Bias Comparisons

The experiments reveal that SASs exhibit more statistical bias on human-generated datasets than on synthetic data, highlighting the importance of using real-world data for bias assessment. Additionally, human perception of bias ( $S_h$ ) shows less statistical bias but some confounding bias, indicating discrepancies between automated sentiment scores and human judgment.

Results of Causal Testing

The causal relationships in the datasets were analyzed to validate the hypotheses on bias. The paper computed Weighted Rejection Scores (WRS) and Deconfounding Impact Estimations (DIE) to quantify bias levels in different datasets.

Figure 5: Causal model for rating SASs on HD1.

Figure 6: Causal model for rating SASs on HD2.

Conclusion

The paper demonstrates that the bias in SASs can be more accurately assessed using human-generated data and round-trip translation methods, compared to purely synthetic datasets. Round-trip translation generally decreased bias in human data but increased it in synthetic data. These insights provide valuable guidance for developing SASs with reduced bias, especially for applications in multilingual and diverse settings. This research emphasizes the importance of contextually relevant test data and causal inference techniques in evaluating AI systems for bias.