Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
41 tokens/sec
GPT-4o
60 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
8 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Blind Spots and Biases: Exploring the Role of Annotator Cognitive Biases in NLP (2404.19071v1)

Published 29 Apr 2024 in cs.HC and cs.CL

Abstract: With the rapid proliferation of artificial intelligence, there is growing concern over its potential to exacerbate existing biases and societal disparities and introduce novel ones. This issue has prompted widespread attention from academia, policymakers, industry, and civil society. While evidence suggests that integrating human perspectives can mitigate bias-related issues in AI systems, it also introduces challenges associated with cognitive biases inherent in human decision-making. Our research focuses on reviewing existing methodologies and ongoing investigations aimed at understanding annotation attributes that contribute to bias.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (63)
  1. Persistent anti-muslim bias in large language models. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society, pages 298–306.
  2. A reductions approach to fair classification. In International conference on machine learning, pages 60–69. PMLR.
  3. Fair active learning. Expert Systems with Applications, 199:116981.
  4. Academic accomplices: Practical strategies for research justice. In Companion Publication of the 2019 on Designing Interactive Systems Conference 2019 Companion, pages 353–356.
  5. Characterising and mitigating aggregationbias in crowdsourced toxicity annotations. In Proceedings of the 1st Workshop on Disentangling the Relation between Crowdsourcing and Bias Management. CEUR.
  6. On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency, pages 610–623.
  7. Herbert Bless and Klaus Fiedler. 2014. Social cognition: How individuals construct social reality. Psychology Press.
  8. Language (technology) is power: A critical survey of" bias" in nlp. arXiv preprint arXiv:2005.14050.
  9. Developing a reference protocol for structured expert elicitation in health-care decision-making: a mixed-methods study. Health Technology Assessment (Winchester, England), 25(37):1.
  10. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. Advances in neural information processing systems, 29.
  11. Systematic labeling bias: De-biasing where everyone is wrong. In 2014 22nd International Conference on Pattern Recognition, pages 4417–4422. IEEE.
  12. Optimized pre-processing for discrimination prevention. Advances in neural information processing systems, 30.
  13. Kate Crawford. 2016. Artificial intelligence’s white guy problem. The New York Times, 25(06):5.
  14. Crowdworksheets: Accounting for individual and collective identities underlying crowdsourced dataset annotation. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, pages 2342–2351.
  15. Fairness through awareness. In Proceedings of the 3rd innovations in theoretical computer science conference, pages 214–226.
  16. Carsten Eickhoff. 2014. Crowd-powered experts: Helping surgeons interpret breast cancer images. In Proceedings of the First International Workshop on Gamification for Information Retrieval, pages 53–56.
  17. Carsten Eickhoff. 2018. Cognitive biases in crowdsourcing. In Proceedings of the eleventh ACM international conference on web search and data mining, pages 162–170.
  18. Quality through flow and immersion: gamifying crowdsourced relevance assessments. In Proceedings of the 35th international ACM SIGIR conference on Research and development in information retrieval, pages 871–880.
  19. Daniel Ellsberg. 1961. Risk, ambiguity, and the savage axioms. The quarterly journal of economics, 75(4):643–669.
  20. Certifying and removing disparate impact. arXiv preprint arXiv:1412.3756.
  21. Crowdsourcing and annotating ner for twitter# drift. In LREC, pages 2544–2547.
  22. Ben Green and Lily Hu. 2018. The myth in the methodology: Towards a recontextualization of fairness in machine learning. In Proceedings of the machine learning: the debates workshop.
  23. Algorithmic bias: From discrimination discovery to fairness-aware data mining. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pages 2125–2126.
  24. Equality of opportunity in supervised learning. Advances in neural information processing systems, 29.
  25. Uncertainty and inclusivity in gender bias annotation: An annotation taxonomy and annotated datasets of british english text. In Proceedings of the 4th Workshop on Gender Bias in Natural Language Processing (GeBNLP), pages 30–57.
  26. Investigating and mitigating biases in crowdsourced data. In Companion Publication of the 2021 Conference on Computer Supported Cooperative Work and Social Computing, pages 331–334.
  27. Dirk Hovy and Shrimai Prabhumoye. 2021. Five sources of bias in natural language processing. Language and linguistics compass, 15(8):e12432.
  28. Lilly C Irani and M Six Silberman. 2013. Turkopticon: Interrupting worker invisibility in amazon mechanical turk. In Proceedings of the SIGCHI conference on human factors in computing systems, pages 611–620.
  29. Fairness in learning: Classic and contextual bandits. Advances in neural information processing systems, 29.
  30. Faisal Kamiran and Toon Calders. 2012. Data preprocessing techniques for classification without discrimination. Knowledge and information systems, 33(1):1–33.
  31. Chatgpt needs spade (sustainability, privacy, digital divide, and ethics) evaluation: A review. arXiv preprint arXiv:2305.03123.
  32. Inherent trade-offs in the fair determination of risk scores. arXiv preprint arXiv:1609.05807.
  33. Adaptive sensitive reweighting to mitigate bias in fairness-aware classification. In Proceedings of the 2018 world wide web conference, pages 853–862.
  34. Michelle Seng Ah Lee and Luciano Floridi. 2021. Algorithmic fairness in mortgage lending: from absolute conditions to relational trade-offs. Minds and Machines, 31(1):165–191.
  35. Michelle Seng Ah Lee and Jat Singh. 2021. The landscape and gaps in open source fairness toolkits. In Proceedings of the 2021 CHI conference on human factors in computing systems, pages 1–13.
  36. Out of context: Investigating the bias and fairness concerns of “artificial intelligence as a service”. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, pages 1–17.
  37. Calibrated fairness in bandits. arXiv preprint arXiv:1707.01875.
  38. The evolution of cognitive bias. 724-746 in the handbook of evolutionary psychology, edited by david m. buss.
  39. A survey on bias and fairness in machine learning. ACM computing surveys (CSUR), 54(6):1–35.
  40. Don’t blame the annotator: Bias already starts in the annotation instructions. arXiv preprint arXiv:2205.00415.
  41. Linguistically debatable or just plain wrong? In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 507–511.
  42. On fairness and calibration. Advances in neural information processing systems, 30.
  43. Alexander J Quinn and Benjamin B Bederson. 2011. Human computation: a survey and taxonomy of a growing field. In Proceedings of the SIGCHI conference on human factors in computing systems, pages 1403–1412.
  44. Katharina Reinecke and Krzysztof Z Gajos. 2015. Labinthewild: Conducting large-scale online experiments with uncompensated samples. In Proceedings of the 18th ACM conference on computer supported cooperative work & social computing, pages 1364–1378.
  45. Chatgpt: Bullshit spewer or the end of traditional assessments in higher education? Journal of Applied Learning and Teaching, 6(1).
  46. Re-imagining algorithmic fairness in india and beyond. In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency, pages 315–328.
  47. The risk of racial bias in hate speech detection. In Proceedings of the 57th annual meeting of the association for computational linguistics, pages 1668–1678.
  48. Katie Seaborn and Yeongdae Kim. 2023. “i’m” lost in translation: Pronoun missteps in crowdsourced data sets. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems, pages 1–6.
  49. Fairness and abstraction in sociotechnical systems. In Proceedings of the conference on fairness, accountability, and transparency, pages 59–68.
  50. An investigation of critical issues in bias mitigation techniques. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 1943–1954.
  51. Herbert A Simon. 1957. Models of man; social and rational.
  52. Mitigating gender bias in natural language processing: Literature review. arXiv preprint arXiv:1906.08976.
  53. Harini Suresh and John Guttag. 2021. A framework for understanding sources of harm throughout the machine learning life cycle. In Equity and access in algorithms, mechanisms, and optimization, pages 1–9.
  54. Alaina Talboy and Sandra Schneider. 2022. Reference dependence in bayesian reasoning: Value selection bias, congruence effects, and response prompt sensitivity. Frontiers in Psychology, 13:729285.
  55. Lamda: Language models for dialog applications. arXiv preprint arXiv:2201.08239.
  56. Amos Tversky and Daniel Kahneman. 1974. Judgment under uncertainty: Heuristics and biases: Biases in judgments reveal some heuristics of thinking under uncertainty. science, 185(4157):1124–1131.
  57. Crowdsourcing perceptions of fair predictors for machine learning: A recidivism case study. Proceedings of the ACM on Human-Computer Interaction, 3(CSCW):1–21.
  58. Effect of information presentation on fairness perceptions of machine learning predictors. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, pages 1–13.
  59. Chatgpt: five priorities for research. Nature, 614(7947):224–226.
  60. Sahil Verma and Julia Rubin. 2018. Fairness definitions explained. In Proceedings of the international workshop on software fairness, pages 1–7.
  61. Characterizing and detecting malicious crowdsourcing. In Proceedings of the ACM SIGCOMM 2013 conference on SIGCOMM, pages 537–538.
  62. A survey of human-in-the-loop for machine learning. Future Generation Computer Systems, 135:364–381.
  63. Fairness beyond disparate treatment & disparate impact: Learning classification without disparate mistreatment. In Proceedings of the 26th international conference on world wide web, pages 1171–1180.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (2)
  1. Sanjana Gautam (11 papers)
  2. Mukund Srinath (10 papers)
Citations (3)
View on Semantic Scholar
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets