Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
80 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Toxic Bias: Perspective API Misreads German as More Toxic (2312.12651v3)

Published 19 Dec 2023 in cs.SI

Abstract: Proprietary public APIs play a crucial and growing role as research tools among social scientists. Among such APIs, Google's machine learning-based Perspective API is extensively utilized for assessing the toxicity of social media messages, providing both an important resource for researchers and automatic content moderation. However, this paper exposes an important bias in Perspective API concerning German language text. Through an in-depth examination of several datasets, we uncover intrinsic language biases within the multilingual model of Perspective API. We find that the toxicity assessment of German content produces significantly higher toxicity levels than other languages. This finding is robust across various translations, topics, and data sources, and has significant consequences for both research and moderation strategies that rely on Perspective API. For instance, we show that, on average, four times more tweets and users would be moderated when using the German language compared to their English translation. Our findings point to broader risks associated with the widespread use of proprietary APIs within the computational social sciences.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (26)
  1. Emily A Vogels. The state of online harassment. Pew Research Center, 13:625, 2021.
  2. A new generation of Perspective API: Efficient multilingual character-level transformers. In The 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD’22), pages 3197–3207, 2022.
  3. Using machine learning to reduce toxicity online. https://www.perspectiveapi.com/. Accessed: 2023-10-31.
  4. Perspective API. https://www.perspectiveapi.com/case-studies/. Accessed: 2023-10-31.
  5. RealToxicityPrompts: Evaluating neural toxic degeneration in language models. In The 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP’20), pages 3356–3369, 2020.
  6. Challenges in detoxifying language models. In The 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP’21), pages 2447–2469, 2021.
  7. ConvAI at SemEval-2019 task 6: Offensive language identification and categorization with Perspective and BERT. In Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval’19), pages 571–576, 2019.
  8. Towards measuring adversarial Twitter interactions against candidates in the US midterm elections. In Proceedings of the International AAAI Conference on Web and Social Media (ICWSM’20), volume 14, pages 272–282, 2020.
  9. Characterizing Twitter users who engage in adversarial interactions against political candidates. In The 2020 CHI Conference on Human Factors in Computing Systems (CHI’20), pages 1–13, 2020.
  10. The fabrics of machine moderation: Studying the technical, normative, and organizational structure of Perspective API. Big Data & Society, 8(2), 2021.
  11. Paul Friedl. Dis/similarities in the design and development of legal and algorithmic normative systems: The case of Perspective API. Law, Innovation and Technology, 15(1):25–59, 2023.
  12. Reading in-between the lines: An analysis of Dissenter. In Proceedings of the ACM Internet Measurement Conference (IMC’20), pages 133–146, 2020.
  13. On the globalization of the QAnon conspiracy theory through Telegram. In Proceedings of the 15th ACM Web Science Conference (WebSci’23), pages 75–85, 2023.
  14. Toxic comments reduce the activity of volunteer editors on Wikipedia. arXiv preprint:2304.13568, 2023.
  15. Leveraging multilingual transformers for hate speech detection. In The 12th meeting of Forum for Information Retrieval Evaluation (FIRE’20), 2020.
  16. Collective moderation of hate, toxicity, and extremity in online discussions. arXiv preprint:2303.00357, 2023.
  17. Catalyst of hate? Ethnic insulting on YouTube in the aftermath of terror attacks in France, Germany and the United Kingdom 2014–2017. Journal of Ethnic and Migration Studies, 49(2):535–553, 2023.
  18. Analyzing and addressing the difference in toxicity prediction between different comments with same semantic meaning in Google’s Perspective API. In The 7th International Conference on ICT for Sustainable Development (ICT4SD’22), pages 455–464, 2022.
  19. Introducing an abusive language classification framework for Telegram to investigate the German hater community. In Proceedings of the International AAAI Conference on Web and Social Media (ICWSM’22), volume 16, pages 1133–1144, 2022.
  20. How toxic is antisemitism? Potentials and limitations of automated toxicity scoring for antisemitic online content. arXiv preprint:2310.04465, 2023.
  21. VaccinEU: COVID-19 vaccine conversations on Twitter in French, German and Italian. Proceedings of the International AAAI Conference on Web and Social Media (ICWSM’22), 2022.
  22. Misinformation, manipulation, and abuse on social media in the era of COVID-19. Journal of Computational Social Science, 3:271–277, 2020.
  23. Research note: Examining how various social media platforms have responded to COVID-19 misinformation. Harvard Kennedy School Misinformation Review, 2(6):1–25, 2021.
  24. Argos Translate. http://www.argosopentech.com/. Accessed: 2023-09-21.
  25. Perspective API threshold. https://developers.perspectiveapi.com/s/about-the-api-score?language=en_US. Accessed: 2023-09-21.
  26. On the challenges of using black-box apis for toxicity evaluation in research. In ICLR 2023 Workshop on Trustworthy and Reliable Large-Scale Machine Learning Models, 2023.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (6)
  1. Gianluca Nogara (8 papers)
  2. Francesco Pierri (44 papers)
  3. Stefano Cresci (40 papers)
  4. Luca Luceri (52 papers)
  5. Petter Törnberg (9 papers)
  6. Silvia Giordano (24 papers)
Citations (14)
View on Semantic Scholar