Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
41 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
41 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Investigating Practices and Opportunities for Cross-functional Collaboration around AI Fairness in Industry Practice (2306.06542v1)

Published 10 Jun 2023 in cs.HC, cs.CY, and cs.LG

Abstract: An emerging body of research indicates that ineffective cross-functional collaboration -- the interdisciplinary work done by industry practitioners across roles -- represents a major barrier to addressing issues of fairness in AI design and development. In this research, we sought to better understand practitioners' current practices and tactics to enact cross-functional collaboration for AI fairness, in order to identify opportunities to support more effective collaboration. We conducted a series of interviews and design workshops with 23 industry practitioners spanning various roles from 17 companies. We found that practitioners engaged in bridging work to overcome frictions in understanding, contextualization, and evaluation around AI fairness across roles. In addition, in organizational contexts with a lack of resources and incentives for fairness work, practitioners often piggybacked on existing requirements (e.g., for privacy assessments) and AI development norms (e.g., the use of quantitative evaluation metrics), although they worry that these tactics may be fundamentally compromised. Finally, we draw attention to the invisible labor that practitioners take on as part of this bridging and piggybacking work to enact interdisciplinary collaboration for fairness. We close by discussing opportunities for both FAccT researchers and AI practitioners to better support cross-functional collaboration for fairness in the design and development of AI systems.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (99)
  1. Andrew Abbott. 2014. The system of professions: An essay on the division of expert labor. University of Chicago press.
  2. IBM Resaerch Trusted AI. 2021. AIF360 API. (2021). https://aif360.mybluemix.net/
  3. How Teams Communicate about the Quality of ML Models: A Case Study at an International Technology Company. Proceedings of the ACM on Human-Computer Interaction 5, GROUP (2021), 1–24.
  4. Software engineering for machine learning: A case study. In 2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP). IEEE, 291–300.
  5. Guidelines for human-AI interaction. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–13.
  6. Jacqui Ayling and Adriane Chapman. 2022. Putting AI ethics to work: are the tools fit for purpose? AI and Ethics 2, 3 (2022), 405–429.
  7. Kenneth A Bamberger and Deirdre K Mulligan. 2011. Privacy on the Books and on the Ground. Stanford Law Review (2011), 247–315.
  8. Solon Barocas and Andrew D Selbst. 2016. Big data’s disparate impact. Calif. L. Rev. 104 (2016), 671.
  9. Symphony: Composing interactive interfaces for machine learning. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. 1–14.
  10. Sarah Bird. 2020. Fairlearn API. https://fairlearn.github.io/v0.5.0/api_reference/fairlearn.datasets.html
  11. Fairlearn: A toolkit for assessing and improving fairness in AI. Technical Report MSR-TR-2020-32. Microsoft. https://www.microsoft.com/en-us/research/publication/fairlearn-a-toolkit-for-assessing-and-improving-fairness-in-ai/
  12. The values encoded in machine learning research. In 2022 ACM Conference on Fairness, Accountability, and Transparency. 173–184.
  13. Language (technology) is power: A critical survey of” bias” in nlp. arXiv preprint arXiv:2005.14050 (2020).
  14. Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qual. Res. Psychol. 3, 2 (Jan. 2006), 77–101.
  15. Howard Brown and Timothy J. Larson. 1998. Making business integration work: A survival strategy for EHS managers. Environmental Quality Management 7 (1998), 1–8.
  16. Helen Burnie. 2016. Piggybacking on sustainability. Practical Literacy: The Early and Primary Years 21, 3 (2016), 35–38.
  17. Discovering and validating ai errors with crowdsourced failure reports. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–22.
  18. FairVis: Visual analytics for discovering intersectional bias in machine learning. In 2019 IEEE Conference on Visual Analytics Science and Technology (VAST). IEEE, 46–56.
  19. What Did My AI Learn? How Data Scientists Make Sense of Model Behavior. ACM Transactions on Computer-Human Interaction (2022).
  20. Purposive sampling: complex or simple? Research case examples. Journal of research in Nursing 25, 8 (2020), 652–661.
  21. Identity Claims that Underlie Ethical Awareness and Action. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–13.
  22. Sex and gender differences and biases in artificial intelligence for biomedicine and healthcare. NPJ digital medicine 3, 1 (2020), 81.
  23. Colab. 2020. Welcome To Colaboratory. https://colab.research.google.com/
  24. Assessing and addressing algorithmic bias in practice. Interactions 25, 6 (2018), 58–63.
  25. Kate Crawford. 2017. The trouble with bias. keynote at neurips. (2017).
  26. Understanding Practices, Challenges, and Opportunities for User-Engaged Algorithm Auditing in Industry Practice. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–18.
  27. Exploring how machine learning practitioners (try to) use fairness toolkits. In 2022 ACM Conference on Fairness, Accountability, and Transparency. 473–484.
  28. Toward User-Driven Algorithm Auditing: Investigating users’ strategies for uncovering harmful algorithmic behavior. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. 1–19.
  29. Thriving, Not Just Surviving in Changing Times: How Sustainability, Agility and Digitalization Intertwine with Organizational Resilience. Sustainability (2021).
  30. Shelley Evenson. 2006. Directed storytelling: Interpreting experience for design. Design Studies: Theory and research in graphic design (2006), 231–240.
  31. EY. 2021. EY’s Trust Score: Defining business priorities for long-term value. (2021). https://www.ey.com/en_us/consulting/trusted-ai-platform
  32. Diana Forsythe. 2001. Studying those who study us: An anthropologist in the world of artificial intelligence. Stanford University Press.
  33. The (im) possibility of fairness: Different value systems require different mechanisms for fair decision making. Commun. ACM 64, 4 (2021), 136–143.
  34. Timnit Gebru. 2021. Hierarchy of Knowledge in Machine Learning and Related Fields and Its Consequences. https://www.youtube.com/watch?v=OL3DowBM9uc
  35. Gamestorming: A playbook for innovators, rulebreakers, and changemakers. ” O’Reilly Media, Inc.”.
  36. Ben Green. 2021. The contestation of tech ethics: A sociotechnical approach to technology ethics in practice. Journal of Social Computing 2, 3 (2021), 209–225.
  37. How does usable security (not) end up in software products? results from a qualitative interview study. In 43rd IEEE Symposium on Security and Privacy, IEEE S&P. 22–26.
  38. Melissa Heikkilä. 2022. Responsible AI has a burnout problem. https://www.technologyreview.com/2022/10/28/1062332/responsible-ai-has-a-burnout-problem/
  39. Cross-Functional Teams: Good Concept, Poor Implementation! Journal of Product Innovation Management 10 (1993), 216–229.
  40. Improving fairness in machine learning systems: What do industry practitioners need?. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–16.
  41. Evaluation Gaps in Machine Learning Practice. 2022 ACM Conference on Fairness, Accountability, and Transparency (2022).
  42. The global landscape of AI ethics guidelines. Nature Machine Intelligence 1, 9 (2019), 389–399.
  43. Jupyter. 2020. Jypyter: Free software, open standards, and web services for interactive computing across all programming languages. https://jupyter.org/
  44. Kenneth B Kahn. 1996. Interdepartmental integration: a definition with implications for product development performance. Journal of product innovation management 13, 2 (1996), 137–151.
  45. Interpreting Interpretability: Understanding Data Scientists’ Use of Interpretability Tools for Machine Learning. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (2020).
  46. Inherent trade-offs in the fair determination of risk scores. arXiv preprint arXiv:1609.05807 (2016).
  47. Sean Kross and Philip Guo. 2021. Orienting, framing, bridging, magic, and counseling: How data scientists navigate the outer loop of client collaborations in industry and academia. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–28.
  48. Model Sketching: Centering Concepts in Early-Stage Machine Learning Model Design. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–24.
  49. How HCI Adopts Service Design: Unpacking current perceptions and scopes of service design in HCI and identifying future opportunities. In CHI Conference on Human Factors in Computing Systems. 1–14.
  50. Formalising trade-offs beyond algorithmic fairness: lessons from ethical philosophy and welfare economics. AI and Ethics 1, 4 (2021), 529–544.
  51. Michelle Seng Ah Lee and Jat Singh. 2021. The landscape and gaps in open source fairness toolkits. In Proceedings of the 2021 CHI conference on human factors in computing systems. 1–13.
  52. Out of Context: Investigating the Bias and Fairness Concerns of “Artificial Intelligence as a Service”. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17.
  53. Coconut: An IDE plugin for developing privacy-friendly apps. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 2, 4 (2018), 1–35.
  54. Designerly Understanding: Information Needs for Model Transparency to Support Design Ideation for AI-Powered User Experience. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21.
  55. Barbara Liskov. 1987. Keynote address-data abstraction and hierarchy. In Addendum to the proceedings on Object-oriented programming systems, languages and applications (Addendum). 17–34.
  56. Assessing the Fairness of AI Systems: AI Practitioners’ Processes, Challenges, and Needs for Support. Proceedings of the ACM on Human-Computer Interaction 6, CSCW1 (2022), 1–26.
  57. Need for Organizational Performance Metrics to Support Fairness in AI. Fair and Responsible AI Workshop at the 2020 CHI Conference on Human Factors in Computing Systems (2020), 1–14.
  58. Co-designing checklists to understand organizational challenges and opportunities around fairness in ai. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–14.
  59. How data scientistswork together with domain experts in scientific collaborations: To find the right answer or to ask the right question? Proceedings of the ACM on Human-Computer Interaction 3, GROUP (2019), 1–23.
  60. Owning ethics: Corporate logics, silicon valley, and the institutionalization of ethics. Social Research: An International Quarterly 86, 2 (2019), 449–476.
  61. fAIlureNotes: Supporting Designers in Understanding the Limits of AI Models for Computer Vision Tasks. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–19.
  62. How data science workers work with data: Discovery, capture, curation, design, creation. In Proceedings of the 2019 CHI conference on human factors in computing systems. 1–15.
  63. Dawn Nafus and Jamie Sherman. 2014. This One Does Not Go Up to 11: The Quantified Self Movement as an Alternative Big Data Practice.
  64. Collaboration Challenges in Building ML-Enabled Systems: Communication, Documentation, Engineering, and Process. arXiv preprint arXiv:2110.10234 (2021).
  65. Dissecting racial bias in an algorithm used to manage the health of populations. Science 366, 6464 (2019), 447–453.
  66. Facilitating knowledge sharing from domain experts to data scientists for building nlp models. In 26th International Conference on Intelligent User Interfaces. 585–596.
  67. Samir Passi and Solon Barocas. 2019. Problem formulation and fairness. In Proceedings of the Conference on Fairness, Accountability, and Transparency. 39–48.
  68. Samir Passi and Steven J Jackson. 2018. Trust in data science: Collaboration, translation, and accountability in corporate data science projects. Proceedings of the ACM on Human-Computer Interaction 2, CSCW (2018), 1–28.
  69. Ten simple rules for running interactive workshops. PLoS computational biology 10, 2 (2014), e1003485.
  70. How ai developers overcome communication challenges in a multidisciplinary team: A case study. Proceedings of the ACM on Human-Computer Interaction 5, CSCW1 (2021), 1–25.
  71. Saving face: Investigating the ethical concerns of facial recognition auditing. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society. 145–151.
  72. Where responsible AI meets reality: Practitioner perspectives on enablers for shifting organizational practices. Proceedings of the ACM on Human-Computer Interaction 5, CSCW1 (2021), 1–23.
  73. Towards Fairness in Practice: A Practitioner-Oriented Rubric for Evaluating Fair ML Toolkits. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–13.
  74. Thomas N Robinson. 2010. Save the world, prevent obesity: piggybacking on existing social and ideological movements. Obesity 18, n1s (2010), S17.
  75. Healthsheet: development of a transparency artifact for health datasets. In 2022 ACM Conference on Fairness, Accountability, and Transparency. 1943–1961.
  76. W Richard Scott and Gerald F Davis. 2015. Organizations and organizing: Rational, natural and open systems perspectives. Routledge.
  77. Fairness and abstraction in sociotechnical systems. In Proceedings of the conference on fairness, accountability, and transparency. 59–68.
  78. Value cards: An educational toolkit for teaching social impacts of machine learning through deliberation. In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency. 850–861.
  79. Everyday algorithm auditing: Understanding the power of everyday users in surfacing harmful algorithmic behaviors. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–29.
  80. Ben Shneiderman. 2020. Human-Centered Artificial Intelligence: Reliable, Safe & Trustworthy. International Journal of Human–Computer Interaction 36 (2020), 495 – 504.
  81. Mario Luis Small and Jessica McCrory Calarco. 2022. Qualitative literacy: A guide to evaluating ethnographic and interview research. Univ of California Press.
  82. REAL ML: Recognizing, Exploring, and Articulating Limitations of Machine Learning Research. In 2022 ACM Conference on Fairness, Accountability, and Transparency. 587–597.
  83. X Michael Song and Mark E Parry. 1997. A cross-national comparative study of new product development processes: Japan and the United States. Journal of marketing 61, 2 (1997), 1–18.
  84. Susan Leigh Star and Anselm Strauss. 1999. Layers of silence, arenas of voice: The ecology of visible and invisible work. Computer supported cooperative work 8, 1-2 (1999), 9–30.
  85. Solving Separation-of-Concerns Problems in Collaborative Design of Human-AI Systems through Leaky Abstractions. In CHI Conference on Human Factors in Computing Systems. 1–21.
  86. Towards a process model for co-creating AI experiences. In Designing Interactive Systems Conference 2021. 1529–1543.
  87. Privacy champions in software teams: understanding their motivations, strategies, and challenges. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–15.
  88. Kush R Varshney. 2019. Trustworthy machine learning and artificial intelligence. XRDS: Crossroads, The ACM Magazine for Students 25, 3 (2019), 26–29.
  89. Ari Ezra Waldman. 2021. Industry unbound: The inside story of privacy, data, and corporate power. Cambridge University Press.
  90. Measuring representational harms in image captioning. In 2022 ACM Conference on Fairness, Accountability, and Transparency. 324–335.
  91. Designing Responsible AI: Adaptations of UX Practice to Meet Responsible AI Challenges. (2023).
  92. Richmond Y Wong. 2021. Tactics of Soft Resistance in User Experience Professionals’ Values Work. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–28.
  93. Seeing Like a Toolkit: How Toolkits Envision the Work of AI Ethics. arXiv preprint arXiv:2202.08792 (2022).
  94. Richmond Y Wong and Tonya Nguyen. 2021. Timelines: A world-building activity for values advocacy. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–15.
  95. Re-examining whether, why, and how human-AI interaction is uniquely difficult to design. In Proceedings of the 2020 chi conference on human factors in computing systems. 1–13.
  96. How Experienced Designers of Enterprise Applications Engage AI as a Design Material. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. 1–13.
  97. Investigating How Practitioners Use Human-AI Guidelines: A Case Study on the People+ AI Guidebook. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–13.
  98. Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–21.
  99. How do data science workers collaborate? roles, workflows, and tools. Proceedings of the ACM on Human-Computer Interaction 4, CSCW1 (2020), 1–23.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (6)
  1. Wesley Hanwen Deng (15 papers)
  2. Nur Yildirim (6 papers)
  3. Monica Chang (1 paper)
  4. Motahhare Eslami (27 papers)
  5. Ken Holstein (3 papers)
  6. Michael Madaio (15 papers)
Citations (32)