Going public: the role of public participation approaches in commercial AI labs (2306.09871v1)
Abstract: In recent years, discussions of responsible AI practices have seen growing support for "participatory AI" approaches, intended to involve members of the public in the design and development of AI systems. Prior research has identified a lack of standardised methods or approaches for how to use participatory approaches in the AI development process. At present, there is a dearth of evidence on attitudes to and approaches for participation in the sites driving major AI developments: commercial AI labs. Through 12 semi-structured interviews with industry practitioners and subject-matter experts, this paper explores how commercial AI labs understand participatory AI approaches and the obstacles they have faced implementing these practices in the development of AI systems and research. We find that while interviewees view participation as a normative project that helps achieve "societally beneficial" AI systems, practitioners face numerous barriers to embedding participatory approaches in their companies: participation is expensive and resource intensive, it is "atomised" within companies, there is concern about exploitation, there is no incentive to be transparent about its adoption, and it is complicated by a lack of clear context. These barriers result in a piecemeal approach to participation that confers no decision-making power to participants and has little ongoing impact for AI labs. This papers contribution is to provide novel empirical research on the implementation of public participation in commercial AI labs, and shed light on the current challenges of using participatory approaches in this context.
- Shana Agid ““…it’s your project, but it’s not necessarily your work…”: infrastructuring, situatedness, and designing relational practice” In Proceedings of the 14th Participatory Design Conference: Full papers - Volume 1, PDC ’16 New York, NY, USA: Association for Computing Machinery, 2016, pp. 81–90 DOI: 10.1145/2940299.2940317
- Sherry R. Arnstein “A Ladder Of Citizen Participation” Publisher: Routledge _eprint: https://doi.org/10.1080/01944366908977225 In Journal of the American Institute of Planners 35.4, 1969, pp. 216–224 DOI: 10.1080/01944366908977225
- “A Policymaker’s Guide to the “Techlash”—What It Is and Why It’s a Threat to Growth and Progress — ITIF”, 2019 URL: https://itif.org/publications/2019/10/28/policymakers-guide-techlash/
- “Reading Race: AI Recognises Patient’s Racial Identity In Medical Images” In The Lancet Digital Health 4.6, 2022, pp. e406–e414 DOI: 10.1016/S2589-7500(22)00063-2
- “Crowdsourcing Impacts: Exploring the Utility of Crowds for Anticipating Societal Impacts of Algorithmic Decision Making” In Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society, 2022, pp. 56–67 DOI: 10.1145/3514094.3534145
- Aleks Berditchevskaia, Eirini Malliaraki and Kathy Peach “Participatory AI for humanitarian innovation: a briefing paper”, 2021 URL: https://www.nesta.org.uk/report/participatory-ai-humanitarian-innovation-briefing-paper/
- Elettra Bietti “From ethics washing to ethics bashing: a view on tech ethics from within moral philosophy” In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, FAT* ’20 New York, NY, USA: Association for Computing Machinery, 2020, pp. 210–219 DOI: 10.1145/3351095.3372860
- “Power to the People? Opportunities and Challenges for Participatory AI”, 2022 DOI: 10.1145/3551624.3555290
- “The Values Encoded in Machine Learning Research” arXiv, 2022 arXiv: http://arxiv.org/abs/2106.15590
- BIT “Deliberative democracy in action”, 2022 URL: https://www.bi.team/blogs/deliberative-democracy-in-action/
- “Deliberation and Inclusion: Vehicles for Increasing Trust in UK Public Governance?” Publisher: SAGE Publications Ltd STM In Environment and Planning C: Government and Policy 19.4, 2001, pp. 501–513 DOI: 10.1068/c6s
- “Tech Worker Organizing for Power and Accountability” In 2022 ACM Conference on Fairness, Accountability, and Transparency Seoul Republic of Korea: ACM, 2022, pp. 452–463 DOI: 10.1145/3531146.3533111
- “Envisioning Communities: A Participatory Approach Towards AI for Social Good” In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society, 2021, pp. 425–436 DOI: 10.1145/3461702.3462612
- Mark Bovens “Analysing and Assessing Public Accountability. A Conceptual Framework” In European Law Journal, 2007, pp. 37 URL: https://onlinelibrary.wiley.com/doi/abs/10.1111/j.1468-0386.2007.00378.x
- “Unpacking the Notion of Participation in Participatory Design” In Computer Supported Cooperative Work (CSCW) 25.6, 2016, pp. 425–475 DOI: 10.1007/s10606-016-9259-4
- “Using thematic analysis in psychology” In Qualitative Research in Psychology 3.2, 2006, pp. 77–101 DOI: 10.1191/1478088706qp063oa
- Ellie Brodie, Eddie Cowling and Nina Nissen “Understanding participation:” In An introduction, 2009, pp. 50
- “Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification”, 2018, pp. 15
- Vivien Burr “An Introduction to Social Constructionism” Routledge, 2006 DOI: 10.4324/9780203133026
- “The Limits of Global Inclusion in AI Development” arXiv, 2021 arXiv: http://arxiv.org/abs/2102.01265
- “Deep reinforcement learning from human preferences” Publisher: arXiv Version Number: 4, 2017 DOI: 10.48550/ARXIV.1706.03741
- Frances Cleaver “Paradoxes of participation: questioning participatory approaches to development” In Journal of International Development 11.4, 1999, pp. 597–612 DOI: 10.1002/(SICI)1099-1328(199906)11:4¡597::AID-JID610¿3.0.CO;2-Q
- Jennifer Cobbe, Michael Veale and Jatinder Singh “Understanding accountability in algorithmic supply chains”, 2023 URL: https://papers.ssrn.com/abstract=4430778
- “Accountability in an Algorithmic Society: Relationality, Responsibility, and Robustness in Machine Learning” In arXiv:2202.05338 [cs], 2022 DOI: 10.1145/3531146.3533150
- Andrea Cornwall “Unpacking ‘Participation’: models, meanings and practices” In Community Development Journal 43.3, 2008, pp. 269–283 DOI: 10.1093/cdj/bsn010
- Sasha Costanza-Chock “Design justice: community-led practices to build the worlds we need”, Information policy Cambridge, MA: The MIT Press, 2020
- Nick Couldry and Ulises Ali Mejias “The decolonial turn in data and technology research: what is at stake and where is it heading?” Publisher: Routledge _eprint: https://doi.org/10.1080/1369118X.2021.1986102 In Information, Communication & Society 0.0, 2021, pp. 1–17 DOI: 10.1080/1369118X.2021.1986102
- “Stakeholder Participation in AI: Beyond ”Add Diverse Stakeholders and Stir””, 2021 URL: http://arxiv.org/abs/2111.01122
- Paul Dempsey “Access for all: the democratisation of AI”, 2021 URL: https://eandt.theiet.org/content/articles/2021/11/access-for-all-the-democratisation-of-ai/
- “Bringing the People Back In: Contesting Benchmark Machine Learning Datasets” arXiv, 2020 arXiv: http://arxiv.org/abs/2007.07399
- “CrowdWorkSheets: Accounting for Individual and Collective Identities Underlying Crowdsourced Dataset Annotation” In 2022 ACM Conference on Fairness, Accountability, and Transparency, 2022, pp. 2342–2351 DOI: 10.1145/3531146.3534647
- Marc-Antoine Dilhac “Responsible Artificial Intelligence: a Guide for Deliberation — International observatory on the societal impacts of AI and digital technology”, 2021 URL: https://observatoire-ia.ulaval.ca/en/responsible-artificial-intelligence-a-guide-for-deliberation/
- Paul J. DiMaggio and Walter W. Powell “The Iron Cage Revisited: Institutional Isomorphism and Collective Rationality in Organizational Fields” Publisher: [American Sociological Association, Sage Publications, Inc.] In American Sociological Review 48.2, 1983, pp. 147–160 DOI: 10.2307/2095101
- “UX Design Innovation: Challenges for Working with Machine Learning as a Design Material” In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, CHI ’17 New York, NY, USA: Association for Computing Machinery, 2017, pp. 278–288 DOI: 10.1145/3025453.3025739
- “The crisis of democracy and the science of deliberation” Publisher: American Association for the Advancement of Science In Science 363.6432, 2019, pp. 1144–1146 DOI: 10.1126/science.aaw2694
- Virginia Eubanks “Automating Inequality: How High-Tech Tools Profile, Police and Punish the Poor”, 2018
- Chad W Flanders “What Is the Value of Participation?” In OKLAHOMA LAW REVIEW 66, 2013
- Seth Frey, P.M. Krafft and Brian C. Keegan “”This Place Does What It Was Built For”: Designing Digital Institutions for Participatory Change” In Proceedings of the ACM on Human-Computer Interaction 3, 2019, pp. 32:1–32:31 DOI: 10.1145/3359134
- “Artificial Intelligence and Inclusion: Formerly Gang-Involved Youth as Domain Experts for Analyzing Unstructured Twitter Data” Publisher: SAGE Publications Inc In Social Science Computer Review 38.1, 2020, pp. 42–56 DOI: 10.1177/0894439318788314
- “Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned”, 2022
- Michele E. Gilman “Beyond Window Dressing: Public Participation for Marginalized Communities in the Datafied Society”, 2022 URL: https://papers.ssrn.com/abstract=4266250
- Ludo Glimmerveen, Sierk Ybema and Henk Nies “Who Participates in Public Participation? The Exclusionary Effects of Inclusionary Efforts” Publisher: SAGE Publications Inc In Administration & Society 54.4, 2022, pp. 543–574 DOI: 10.1177/00953997211034137
- “Public participation in environmental impact assessment: why, who and how?” In Environmental Impact Assessment Review 43, 2013, pp. 104–111 DOI: 10.1016/j.eiar.2013.06.003
- Jürgen Habermas “Between Facts and Norms: Contributions to a Discourse Theory of Law and Democracy”, Studies in Contemporary German Social Thought Cambridge, MA, USA: MIT Press, 1996
- Christina N. Harrington “The forgotten margins: what is community-based participatory health design telling us?” In Interactions 27.3, 2020, pp. 24–29 DOI: 10.1145/3386381
- Johannes Himmelreich “Against “Democratizing AI”” In AI & SOCIETY, 2022 DOI: 10.1007/s00146-021-01357-z
- “State of AI Report 2022”, 2022 URL: https://www.stateof.ai/
- Kenneth Holstein, Bruce M. McLaren and Vincent Aleven “Co-Designing a Real-Time Classroom Orchestration Tool to Support Teacher–AI Complementarity” In Journal of Learning Analytics 6.2, 2019 DOI: 10.18608/jla.2019.62.3
- Soaad Hossain and Syed Ishtiaque Ahmed “Towards a New Participatory Approach for Designing Artificial Intelligence and Data-Driven Technologies” arXiv, 2021 DOI: 10.48550/arXiv.2104.04072
- Stephan Hügel and Anna R. Davies “Public participation, engagement, and climate change adaptation: A review of the research literature” _eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1002/wcc.645 In WIREs Climate Change 11.4, 2020, pp. e645 DOI: 10.1002/wcc.645
- IAP2 “Core Values, Ethics, Spectrum – The 3 Pillars of Public Participation - International Association for Public Participation” URL: https://www.iap2.org/page/pillars
- Ada Lovelace Institute, AI Now Institute and Open Government Partnership “Algorithmic accountability for the public sector”, 2021, pp. 70 URL: https://www.opengovpartnership.org/documents/algorithmic-accountability-public-sector
- “The human-computer interaction handbook: fundamentals, evolving technologies, and emerging applications”, Human factors and ergonomics Mahwah, N.J: Lawrence Erlbaum Associates, 2003
- Pratyusha Kalluri “Don’t ask if artificial intelligence is good or fair, ask how it shifts power” In Nature 583.7815, 2020, pp. 169–169 DOI: 10.1038/d41586-020-02003-2
- “Toward situated interventions for algorithmic equity: lessons from the field” In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, FAT* ’20 New York, NY, USA: Association for Computing Machinery, 2020, pp. 45–55 DOI: 10.1145/3351095.3372874
- Donald J. Kochan “The Commenting Power: Agency Accountability through Public Participation” In SSRN Electronic Journal, 2017 DOI: 10.2139/ssrn.3006157
- Alexis Lloyd “Camera Obscura: Beyond the lens of user-centered design”, 2020 URL: https://alexis.medium.com/camera-obscura-beyond-the-lens-of-user-centered-design-631bb4f37594
- “Participatory Problem Formulation for Fairer Machine Learning Through Community Based System Dynamics” In arXiv:2005.07572 [cs, stat], 2020 arXiv: http://arxiv.org/abs/2005.07572
- Shakir Mohamed, Marie-Therese Png and William Isaac “Decolonial AI: Decolonial Theory as Sociotechnical Foresight in Artificial Intelligence” In Philosophy & Technology 33.4, 2020, pp. 659–684 DOI: 10.1007/s13347-020-00405-8
- “Ethics Owners: A New Model of Organizational Responsibility in Data-Driven Technology Companies”, 2020, pp. 74 URL: https://datasociety.net/library/ethics-owners/
- “Assembling Accountability: Algorithmic Impact Assessment for the Public Interest”, 2021 URL: https://datasociety.net/library/assembling-accountability-algorithmic-impact-assessment-for-the-public-interest/
- “Exploring the theory, barriers and enablers for patient and public involvement across health, social care and patient safety: a protocol for a systematic review of reviews” Publisher: British Medical Journal Publishing Group Section: Health services research In BMJ Open 7.10, 2017, pp. e018426 DOI: 10.1136/bmjopen-2017-018426
- OpenAI “Announcing OpenAI’s Bug Bounty Program”, 2023 URL: https://openai.com/blog/bug-bounty-program
- OpenAI “How should AI systems behave, and who should decide?”, 2023 URL: https://openai.com/blog/how-should-ai-systems-behave
- “AI-Based Request Augmentation to Increase Crowdsourcing Participation” In Proceedings of the AAAI Conference on Human Computation and Crowdsourcing 7, 2019, pp. 115–124 DOI: 10.1609/hcomp.v7i1.5282
- “Participatory data stewardship”, 2021 URL: https://www.adalovelaceinstitute.org/report/participatory-data-stewardship/
- “Getting Ourselves Together: Data-centered participatory design research & epistemic burden” In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems Yokohama Japan: ACM, 2021, pp. 1–11 DOI: 10.1145/3411764.3445103
- “From smart products to smart systems”, 2018 URL: https://www2.deloitte.com/content/www/us/en/insights/focus/cognitive-technologies/participatory-design-artificial-intelligence.html
- “Where Responsible AI meets Reality: Practitioner Perspectives on Enablers for shifting Organizational Practices” In Proceedings of the ACM on Human-Computer Interaction 5, 2021, pp. 1–23 DOI: 10.1145/3449081
- “What If I Don’t Like Any Of The Choices? The Limits of Preference Elicitation for Participatory Algorithm Design” In arXiv:2007.06718 [cs], 2020 arXiv: http://arxiv.org/abs/2007.06718
- The RSA “Democratising decisions about technology: a toolkit”, 2019 URL: https://www.thersa.org/reports/democratising-decisions-technology-toolkit
- Jill Russell, Nina Fudge and Trish Greenhalgh “The impact of public involvement in health research: what are we measuring? Why are we measuring it? Should we stop measuring it?” In Research Involvement and Engagement 6.1, 2020, pp. 63 DOI: 10.1186/s40900-020-00239-w
- Henrik Skaug Sætra, Harald Borgebund and Mark Coeckelbergh “Avoid diluting democracy by algorithms” In Nature Machine Intelligence 4.10, 2022, pp. 804–806 DOI: 10.1038/s42256-022-00537-w
- “Algorithmic Tools in Public Employment Services: Towards a Jobseeker-Centric Perspective” In 2022 ACM Conference on Fairness, Accountability, and Transparency Seoul Republic of Korea: ACM, 2022, pp. 2138–2148 DOI: 10.1145/3531146.3534631
- “Democratising AI: Multiple Meanings, Goals, and Methods” arXiv, 2023 DOI: 10.48550/arXiv.2303.12642
- Mona Sloane “To make AI fair, here’s what we must learn to do” In Nature 605.7908, 2022, pp. 9–9 DOI: 10.1038/d41586-022-01202-3
- “Participation is not a Design Fix for Machine Learning” arXiv, 2020 arXiv: http://arxiv.org/abs/2007.02423
- “Why and When Should We Use Public Deliberation?” In Hastings Center Report 42.2, 2012, pp. 17–20 DOI: 10.1002/hast.27
- Jack Stilgoe, Richard Owen and Phil Macnaghten “Developing a framework for responsible innovation” In Research Policy 42.9, 2013, pp. 1568–1580 DOI: 10.1016/j.respol.2013.05.008
- “Towards Intersectional Feminist and Participatory ML: A Case Study in Supporting Feminicide Counterdata Collection” In 2022 ACM Conference on Fairness, Accountability, and Transparency Seoul Republic of Korea: ACM, 2022, pp. 667–678 DOI: 10.1145/3531146.3533132
- Jennifer Wortman Vaughan “Making Better Use of the Crowd: How Crowdsourcing Can Advance Machine Learning Research”, 2017, pp. 46 URL: https://jmlr.org/papers/v18/17-234.html
- Lara Groves (2 papers)
- Aidan Peppin (5 papers)
- Andrew Strait (5 papers)
- Jenny Brennan (7 papers)