Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Farsight: Fostering Responsible AI Awareness During AI Application Prototyping (2402.15350v2)

Published 23 Feb 2024 in cs.HC, cs.AI, cs.CY, and cs.LG

Abstract: Prompt-based interfaces for LLMs have made prototyping and building AI-powered applications easier than ever before. However, identifying potential harms that may arise from AI applications remains a challenge, particularly during prompt-based prototyping. To address this, we present Farsight, a novel in situ interactive tool that helps people identify potential harms from the AI applications they are prototyping. Based on a user's prompt, Farsight highlights news articles about relevant AI incidents and allows users to explore and edit LLM-generated use cases, stakeholders, and harms. We report design insights from a co-design study with 10 AI prototypers and findings from a user study with 42 AI prototypers. After using Farsight, AI prototypers in our user study are better able to independently identify potential harms associated with a prompt and find our tool more useful and usable than existing resources. Their qualitative feedback also highlights that Farsight encourages them to focus on end-users and think beyond immediate harms. We discuss these findings and reflect on their implications for designing AI prototyping experiences that meaningfully engage with AI harms. Farsight is publicly accessible at: https://PAIR-code.github.io/farsight.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (207)
  1. Fatih Kadir Akın. 2022. Awesome ChatGPT Prompts. https://github.com/f/awesome-chatgpt-prompts
  2. Walking the Walk of AI Ethics: Organizational Challenges and the Individualization of Risk among Ethics Entrepreneurs. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency.
  3. Software Engineering for Machine Learning: A Case Study. ICSE (2019). https://doi.org/10.1109/ICSE-SEIP.2019.00042
  4. Guidelines for Human-AI Interaction. CHI (2019). https://doi.org/10.1145/3290605.3300233
  5. How Polymorphic Warnings Reduce Habituation in the Brain: Insights from an fMRI Study. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. https://doi.org/10.1145/2702123.2702322
  6. Anthropic. 2023. Core Views on AI Safety: When, Why, What, and How. https://www.anthropic.com/index/core-views-on-ai-safety
  7. Apple. 2023. Human Interface Guidelines: Machine Learning. https://developer.apple.com/design/human-interface-guidelines/machine-learning
  8. Ai Ethics Statements: Analysis and Lessons Learnt from Neurips Broader Impact Statements. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency.
  9. A General Language Assistant as a Laboratory for Alignment. arXiv 2112.00861 (2021). http://arxiv.org/abs/2112.00861
  10. James Auger. 2013. Speculative Design: Crafting the Speculation. Digital Creativity 24 (2013).
  11. Judgment Call the Game: Using Value Sensitive Design and Design Fiction to Surface Ethical Concerns Related to Technology. In Proceedings of the 2019 on Designing Interactive Systems Conference. https://doi.org/10.1145/3322276.3323697
  12. Designing Disaggregated Evaluations of AI Systems: Choices, Considerations, and Tradeoffs. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society (AIES ’21). https://doi.org/10.1145/3461702.3462610
  13. AI Fairness 360: An Extensible Toolkit for Detecting, Understanding, and Mitigating Unwanted Algorithmic Bias. arXiv 1810.01943 (2018). http://arxiv.org/abs/1810.01943
  14. Detecting Discriminatory Risk through Data Annotation Based on Bayesian Inferences. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency. https://doi.org/10.1145/3442188.3445940
  15. Browser Interfaces and Extended Validation SSL Certificates: An Empirical Study. In Proceedings of the 2009 ACM Workshop on Cloud Computing Security. https://doi.org/10.1145/1655008.1655012
  16. Power to the People? Opportunities and Challenges for Participatory AI. In Equity and Access in Algorithms, Mechanisms, and Optimization. https://doi.org/10.1145/3551624.3555290
  17. Rainer Böhme and Stefan Köpsell. 2010. Trained to Accept?: A Field Experiment on Consent Dialogs. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/1753326.1753689
  18. On the Opportunities and Risks of Foundation Models. arXiv 2108.07258 (2022). http://arxiv.org/abs/2108.07258
  19. D33{}^{3}start_FLOATSUPERSCRIPT 3 end_FLOATSUPERSCRIPT Data-Driven Documents. IEEE TVCG 17 (2011). https://doi.org/10.1109/TVCG.2011.185
  20. Overcoming Failures of Imagination in AI Infused System Development and Deployment. arXiv 2011.13416 (2020). http://arxiv.org/abs/2011.13416
  21. Virginia Braun and Victoria Clarke. 2006. Using Thematic Analysis in Psychology. Qualitative Research in Psychology 3 (2006). https://doi.org/10.1191/1478088706qp063oa
  22. Philip AE Brey. 2012. Anticipatory Ethics for Emerging Technologies. Nanoethics 6 (2012).
  23. José Carlos Brustoloni and Ricardo Villamarín-Salomón. 2007. Improving Security Decisions with Polymorphic and Audited Dialogs. In Proceedings of the 3rd Symposium on Usable Privacy and Security. https://doi.org/10.1145/1280680.1280691
  24. AHA!: Facilitating AI Impact Assessment by Generating Examples of Harms. arXiv 2306.03280 (2023). http://arxiv.org/abs/2306.03280
  25. FAIRVIS: Visual Analytics for Discovering Intersectional Bias in Machine Learning. In 2019 IEEE Conference on Visual Analytics Science and Technology (VAST). https://doi.org/10.1109/VAST47406.2019.8986948
  26. 23 Ways to Nudge: A Review of Technology-Mediated Nudging in Human-Computer Interaction. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3290605.3300733
  27. Quantifying Memorization Across Neural Language Models. arXiv 2202.07646 (2023). http://arxiv.org/abs/2202.07646
  28. Extracting Training Data from Large Language Models. arXiv 2012.07805 (2021). http://arxiv.org/abs/2012.07805
  29. Surveying the Landscape of Ethics-Focused Design Methods. arXiv preprint arXiv:2102.08909 (2021).
  30. Alexandra Chouldechova. 2017. Fair Prediction with Disparate Impact: A Study of Bias in Recidivism Prediction Instruments. Big Data 5 (2017).
  31. Andrew R Chow and Billy Perrigo. 2023. The AI Arms Race Is Changing Everything. https://time.com/6255952/ai-impact-chatgpt-microsoft-google/
  32. Jacob Cohen. 1968. Weighted Kappa: Nominal Scale Agreement Provision for Scaled Disagreement or Partial Credit. Psychological Bulletin 70 (1968). https://doi.org/10.1037/h0026256
  33. Jacob Cohen. 2013. Statistical Power Analysis for the Behavioral Sciences (2nd ed ed.).
  34. A Systematic Review and Thematic Analysis of Community-Collaborative Approaches to Computing Research. In CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3491102.3517716
  35. Disentangling Disagreements on Offensiveness: A Cross-Cultural Study. In The 61st Annual Meeting of the Association for Computational Linguistics.
  36. Stakeholder Participation in AI: Beyond ”Add Diverse Stakeholders and Stir”. arXiv 2111.01122 (2021). http://arxiv.org/abs/2111.01122
  37. The Participatory Turn in AI Design: Theoretical Foundations and the Current State of Practice. In Equity and Access in Algorithms, Mechanisms, and Optimization. https://doi.org/10.1145/3617694.3623261
  38. MasterKey: Automated Jailbreak Across Multiple Large Language Model Chatbots. arXiv 2307.08715 (2023). http://arxiv.org/abs/2307.08715
  39. Whose Ground Truth? Accounting for Individual and Collective Identities Underlying Dataset Annotation. arXiv 2112.04554 (2021). http://arxiv.org/abs/2112.04554
  40. Deque. 2023. Axe DevTools: Digital Accessibility Testing Tools Dev Teams Love. https://www.deque.com/axe/devtools/
  41. Erik Derner and Kristina Batistič. 2023. Beyond the Safeguards: Exploring the Security Risks of ChatGPT. arXiv 2305.08005 (2023). http://arxiv.org/abs/2305.08005
  42. Toxicity in ChatGPT: Analyzing Persona-assigned Language Models. arXiv 2304.05335 (2023). http://arxiv.org/abs/2304.05335
  43. CrowdWorkSheets: Accounting for Individual and Collective Identities Underlying Crowdsourced Dataset Annotation. In 2022 ACM Conference on Fairness, Accountability, and Transparency. https://doi.org/10.1145/3531146.3534647
  44. Report of the Montréal Declaration for a Responsible Development of Artificial Intelligence. (2018). https://doi.org/1866/27795
  45. ‘‘That’s Important, but…”: How Computer Science Researchers Anticipate Unintended Consequences of Their Research Innovations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems.
  46. Doteveryone. 2019. Consequence Scanning – an Agile Practice for Responsible Innovators. https://doteveryone.org.uk/project/consequence-scanning/
  47. Dovetail. 2023. Dovetail: All Your Customer Insights in One Place. https://dovetail.com/
  48. Olive Jean Dunn. 1961. Multiple Comparisons among Means. J. Amer. Statist. Assoc. 56 (1961). https://doi.org/10.1080/01621459.1961.10482090
  49. Anthony Dunne and Fiona Raby. 2013. Speculative Everything: Design, Fiction, and Social Dreaming.
  50. You’ve Been Warned: An Empirical Study of the Effectiveness of Web Browser Phishing Warnings. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/1357054.1357219
  51. Serge Egelman and Stuart Schechter. 2013. The Importance of Being Earnest [in Security Warnings]. In Financial Cryptography and Data Security: 17th International Conference, FC 2013, Okinawa, Japan, April 1-5, 2013, Revised Selected Papers 17.
  52. Seamful XAI: Operationalizing Seamful Design in Explainable AI. arXiv preprint arXiv:2211.06753 (2022).
  53. Improving SSL Warnings: Comprehension and Adherence. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. https://doi.org/10.1145/2702123.2702442
  54. Experimenting at Scale with Google Chrome’s SSL Warning. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/2556288.2557292
  55. Programming without a Programming Language: Challenges and Opportunities for Designing Developer Tools for Prompt Programming. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3544549.3585737
  56. What Do We Teach When We Teach Tech Ethics?: A Syllabi Analysis. In Proceedings of the 51st ACM Technical Symposium on Computer Science Education. https://doi.org/10.1145/3328778.3366825
  57. Batya Friedman. 1996. Value-Sensitive Design. interactions 3 (1996).
  58. Batya Friedman and David Hendry. 2012. The Envisioning Cards: A Toolkit for Catalyzing Humanistic and Technical Imaginations. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/2207676.2208562
  59. A Survey of Value Sensitive Design Methods. Foundations and Trends® in Human–Computer Interaction 11 (2017).
  60. Value Sensitive Design: Theory and Methods. University of Washington technical report 2 (2002).
  61. Predictability and Surprise in Large Generative Models. In 2022 ACM Conference on Fairness, Accountability, and Transparency. https://doi.org/10.1145/3531146.3533229
  62. More Than ”If Time Allows”: The Role of Ethics in AI Education. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society. https://doi.org/10.1145/3375627.3375868
  63. Datasheets for Datasets. arXiv:1803.09010 [cs] (2020). http://arxiv.org/abs/1803.09010
  64. RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models. arXiv 2009.11462 (2020). http://arxiv.org/abs/2009.11462
  65. Improving Alignment of Dialogue Agents via Targeted Human Judgements. arXiv 2209.14375 (2022). http://arxiv.org/abs/2209.14375
  66. Stopping Spyware at the Gate: A User Study of Privacy, Notice and Spyware. In Proceedings of the 2005 Symposium on Usable Privacy and Security - SOUPS ’05. https://doi.org/10.1145/1073001.1073006
  67. Noticing Notice: A Large-Scale Experiment on the Timing of Software License Agreements. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/1240624.1240720
  68. Google. 2015. Lit: Simple Fast Web Components. https://lit.dev/
  69. Google. 2016. Lighthouse. https://github.com/GoogleChrome/lighthouse
  70. Google. 2023a. Google Ai Studio: Prototype with Generative AI. https://aistudio.google.com/app
  71. Google. 2023b. PaLM API: Safety Guidance. https://developers.generativeai.google/guide/safety_guidance
  72. Jury Learning: Integrating Dissenting Voices into Machine Learning Models. In CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3491102.3502004
  73. Grammarly. 2023. Grammarly: Free Writing AI Assistance. https://www.grammarly.com/
  74. Hans W. A. Hanley and Zakir Durumeric. 2023. Machine-Made Media: Monitoring the Mobilization of Machine-Generated Articles on Misinformation and Mainstream News Websites. arXiv 2305.09820 (2023). http://arxiv.org/abs/2305.09820
  75. Deconstructing Community-Based Collaborative Design: Towards More Equitable Participatory Design Engagements. Proceedings of the ACM on Human-Computer Interaction 3 (2019). https://doi.org/10.1145/3359318
  76. It’s Time to Do Something: Mitigating the Negative Impacts of Computing through a Change to the Peer Review Process. arXiv preprint arXiv:2112.09544 (2021).
  77. SUMMIT: Scaling Deep Learning Interpretability by Visualizing Activation and Attribution Summarizations. IEEE TVCG (2019). https://doi.org/10.1109/TVCG.2019.2934659
  78. Improving Fairness in Machine Learning Systems: What Do Industry Practitioners Need?. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3290605.3300830
  79. Planning for Natural Language Failures with the AI Playbook. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3411764.3445735
  80. Human Factors in Model Interpretability: Industry Practices, Challenges, and Needs. Proceedings of the ACM on Human-Computer Interaction 4 (2020). https://doi.org/10.1145/3392878
  81. The White House. 2022. Blueprint for an AI Bill of Rights: Making Automated Systems Work for the American People. https://www.whitehouse.gov/ostp/ai-bill-of-rights
  82. A Call to Alarms: Current State and Future Directions in the Battle against Alarm Fatigue. Journal of Electrocardiology 51 (2018). https://doi.org/10.1016/j.jelectrocard.2018.07.024
  83. Queer In AI: A Case Study in Community-Led Participatory AI. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency. http://arxiv.org/abs/2303.16972
  84. PromptMaker: Prompt-based Prototyping with Large Language Models. In CHI Conference on Human Factors in Computing Systems Extended Abstracts. https://doi.org/10.1145/3491101.3503564
  85. Adapting Security Warnings to Counter Online Disinformation. In 30th USENIX Security Symposium (USENIX Security 21). https://www.usenix.org/conference/usenixsecurity21/presentation/kaiser
  86. Sensible AI: Re-imagining Interpretability and Explainability Using Sensemaking Theory. In 2022 ACM Conference on Fairness, Accountability, and Transparency. https://doi.org/10.1145/3531146.3533135
  87. ProPILE: Probing Privacy Leakage in Large Language Models. arXiv 2307.01881 (2023). http://arxiv.org/abs/2307.01881
  88. Online Survey on Novel Designs for Supporting Self-Reflection and Emotion Regulation in Online News Commenting. In Proceedings of the 25th International Academic Mindtrek Conference. https://doi.org/10.1145/3569219.3569411
  89. Shamika Klassen and Casey Fiesler. 2022. ” Run Wild a Little With Your Imagination” Ethical Speculation in Computing Education with Black Mirror. In Proceedings of the 53rd ACM Technical Symposium on Computer Science Education-Volume 1.
  90. Inherent Trade-Offs in the Fair Determination of Risk Scores. arXiv 1609.05807 (2016). http://arxiv.org/abs/1609.05807
  91. Jupyter Notebooks-a Publishing Format for Reproducible Computational Workflows. 2016 (2016). https://doi.org/10.3233/978-1-61499-649-1-87
  92. Learning Is Not a Spectator Sport: Doing Is Better than Watching for Learning from a MOOC. In Proceedings of the Second (2015) ACM Conference on Learning@ Scale.
  93. GeDi: Generative Discriminator Guided Sequence Generation. In Findings of the Association for Computational Linguistics: EMNLP 2021. https://doi.org/10.18653/v1/2021.findings-emnlp.424
  94. J. Richard Landis and Gary G. Koch. 1977. The Measurement of Observer Agreement for Categorical Data. Biometrics 33 (1977). https://doi.org/10.2307/2529310
  95. Michelle Seng Ah Lee and Jatinder Singh. 2020. The Landscape and Gaps in Open Source Fairness Toolkits. SSRN Electronic Journal (2020). https://doi.org/10.2139/ssrn.3695002
  96. Fei-Fei Li and John Etchemendy. 2022. Annual Report 2022: Stanford Institute for Human-centered Artificial Intelligence. https://hai-annual-report.stanford.edu
  97. Multi-Step Jailbreaking Privacy Attacks on ChatGPT. arXiv 2304.05197 (2023). http://arxiv.org/abs/2304.05197
  98. Q. Vera Liao and Jennifer Wortman Vaughan. 2023. AI Transparency in the Age of LLMs: A Human-Centered Research Roadmap. arXiv 2306.01941 (2023). http://arxiv.org/abs/2306.01941
  99. Examining Responsibility and Deliberation in AI Impact Statements and Ethics Reviews. In Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society. https://doi.org/10.1145/3514094.3534155
  100. Trustworthy LLMs: A Survey and Guideline for Evaluating Large Language Models’ Alignment. arXiv 2308.05374 (2023). http://arxiv.org/abs/2308.05374
  101. News from Generative Artificial Intelligence Is Believed Less. In 2022 ACM Conference on Fairness, Accountability, and Transparency. https://doi.org/10.1145/3531146.3533077
  102. Scott M. Lundberg and Su-In Lee. 2017. A Unified Approach to Interpreting Model Predictions. In Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS’17). https://doi.org/10.48550/arXiv.1705.07874
  103. Assessing the Fairness of AI Systems: AI Practitioners’ Processes, Challenges, and Needs for Support. Proceedings of the ACM on Human-Computer Interaction 6 (2022). https://doi.org/10.1145/3512899
  104. Co-Designing Checklists to Understand Organizational Challenges and Opportunities around Fairness in AI. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3313831.3376445
  105. Tambiama André Madiega. 2021. Artificial Intelligence Act. European Parliament: European Parliamentary Research Service (2021). https://artificialintelligenceact.eu
  106. On the Educational Impact of ChatGPT: Is Artificial Intelligence Ready to Obtain a University Degree?, In Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education V. 1. arXiv 2303.11146. https://doi.org/10.1145/3587102.3588827
  107. H. B. Mann and D. R. Whitney. 1947. On a Test of Whether One of Two Random Variables Is Stochastically Larger than the Other. The Annals of Mathematical Statistics 18 (1947). https://doi.org/10.1214/aoms/1177730491
  108. The Potential of Generative AI for Personalized Persuasion at Scale. Preprint. PsyArXiv. https://doi.org/10.31234/osf.io/rn97c
  109. Using Data Type Based Security Alert Dialogs to Raise Online Security Awareness. In Proceedings of the Seventh Symposium on Usable Privacy and Security. https://doi.org/10.1145/2078827.2078830
  110. Kenneth O. McGraw and S. P. Wong. 1992. A Common Language Effect Size Statistic. Psychological Bulletin 111 (1992). https://doi.org/10.1037/0033-2909.111.2.361
  111. Sean McGregor. 2020. Preventing Repeated Real World AI Failures by Cataloging Incidents: The AI Incident Database. arXiv 2011.08512 (2020). http://arxiv.org/abs/2011.08512
  112. Marry L. McHugh. 2012. Interrater Reliability: The Kappa Statistic. Biochemia Medica (2012). https://doi.org/10.11613/BM.2012.031
  113. UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction. arXiv:1802.03426 (2020). http://arxiv.org/abs/1802.03426
  114. MDN. 2011. WebGL: 2D and 3D Graphics for the Web - Web APIs. https://developer.mozilla.org/en-US/docs/Web/API/WebGL_API
  115. MDN. 2021. Web Components - Web APIs. https://developer.mozilla.org/en-US/docs/Web/API/Web_components
  116. Sharan B Merriam et al. 2002. Introduction to Qualitative Research. Qualitative research in practice: Examples for discussion and analysis 1 (2002).
  117. Meta. 2023. Llama 2: Responsible Use Guide. https://ai.meta.com/llama-project/responsible-use-guide
  118. Between Subjectivity and Imposition: Power Dynamics in Data Annotation for Computer Vision. Proceedings of the ACM on Human-Computer Interaction 4 (2020). https://doi.org/10.1145/3415186
  119. Microsoft. 2020. Responsible AI Toolbox. Microsoft. https://github.com/microsoft/responsible-ai-toolbox
  120. Microsoft. 2022a. Harms Modeling - Azure Application Architecture Guide. https://learn.microsoft.com/en-us/azure/architecture/guide/responsible-innovation/harms-modeling/
  121. Microsoft. 2022b. Microsoft Responsible AI Impact Assessment Guide. (2022). https://aka.ms/RAIImpactAssessmentGuidePDF
  122. Model Cards for Model Reporting. In Proceedings of the Conference on Fairness, Accountability, and Transparency. https://doi.org/10.1145/3287560.3287596
  123. Dealing with Disagreements: Looking beyond the Majority Vote in Subjective Annotations. Transactions of the Association for Computational Linguistics 10 (2022). https://doi.org/10.1162/tacl_a_00449
  124. Visual Auditor: Interactive Visualization for Detection and Summarization of Model Biases. In VIS. https://doi.org/10.1109/VIS54862.2022.00018
  125. Delegate the Smartphone User? Security Awareness in Smartphone Platforms. Computers & Security 34 (2013). https://doi.org/10.1016/j.cose.2012.11.004
  126. Anticipatory Ethics and the Role of Uncertainty. arXiv preprint arXiv:2011.13170 (2020).
  127. Unpacking the Expressed Consequences of AI Research in Broader Impact Statements. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society.
  128. InterpretML: A Unified Framework for Machine Learning Interpretability. arXiv (2019). http://arxiv.org/abs/1909.09223
  129. Donald A. Norman and Stephen W. Draper. 1986. User Centered System Design: New Perspectives on Human-Computer Interaction.
  130. Bound by the Bounty: Collaboratively Shaping Evaluation Processes for Queer AI Harms. arXiv 2307.10223 (2023). http://arxiv.org/abs/2307.10223
  131. Cathy O’Neil and Hanna Gunn. 2020. Near-Term Artificial Intelligence and the Ethical Matrix. https://doi.org/10.1093/oso/9780190905033.003.0009
  132. OpenAI. 2023a. GPT-4 Technical Report. arXiv 2303.08774 (2023). http://arxiv.org/abs/2303.08774
  133. OpenAI. 2023b. OpenAI Playground. https://platform.openai.com/playground
  134. Google PAIR. 2019. People + AI Guidebook. https://pair.withgoogle.com/guidebook
  135. On the Risk of Misinformation Pollution with Large Language Models. arXiv 2305.13661 (2023). http://arxiv.org/abs/2305.13661
  136. Ellie Pavlick and Tom Kwiatkowski. 2019. Inherent Disagreements in Human Textual Inferences. Transactions of the Association for Computational Linguistics 7 (2019). https://doi.org/10.1162/tacl_a_00293
  137. Red Teaming Language Models with Language Models. arXiv 2202.03286 (2022). http://arxiv.org/abs/2202.03286
  138. A Framework to Assess (Dis)Agreement Among Diverse Rater Groups. arXiv 2311.05074 (2023). http://arxiv.org/abs/2311.05074
  139. Engaging Students with Instructor Solutions in Online Programming Homework. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3313831.3376857
  140. Institutionalizing Ethics in AI through Broader Impact Requirements. Nature Machine Intelligence 3 (2021).
  141. You Can’t Sit With Us: Exclusionary Pedagogy in AI Ethics Education. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency. https://doi.org/10.1145/3442188.3445914
  142. Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. https://doi.org/10.1145/3351095.3372873
  143. Where Responsible AI Meets Reality: Practitioner Perspectives on Enablers for Shifting Organizational Practices. Proceedings of the ACM on Human-Computer Interaction 5 (2021). https://doi.org/10.1145/3449081
  144. Supporting Human-AI Collaboration in Auditing LLMs with LLMs. In Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society. https://doi.org/10.1145/3600211.3604712
  145. An Experience Sampling Study of User Reactions to Browser Warnings in the Field. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3173574.3174086
  146. E.M. Reingold and J.S. Tilford. 1981. Tidier Drawings of Trees. IEEE Transactions on Software Engineering SE-7 (1981). https://doi.org/10.1109/TSE.1981.234519
  147. An Investigation of Phishing Awareness and Education over Time: When and How to Best Remind Users. In Sixteenth Symposium on Usable Privacy and Security (SOUPS 2020). https://www.usenix.org/conference/soups2020/presentation/reinheimer
  148. Karen Renaud and Marc Dupuis. 2019. Cyber Security Fear Appeals: Unexpectedly Complicated. In Proceedings of the New Security Paradigms Workshop. https://doi.org/10.1145/3368860.3368864
  149. Marco Tulio Ribeiro and Scott Lundberg. 2022. Adaptive Testing and Debugging of NLP Models. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). https://doi.org/10.18653/v1/2022.acl-long.230
  150. ”Why Should I Trust You?”: Explaining the Predictions of Any Classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. https://doi.org/10.1145/2939672.2939778
  151. Beyond Accuracy: Behavioral Testing of NLP Models with CheckList. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. https://doi.org/10.18653/v1/2020.acl-main.442
  152. From Plane Crashes to Algorithmic Harm: Applicability of Safety Engineering Frameworks for Responsible ML. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3544548.3581407
  153. Angler: Helping Machine Translation Practitioners Prioritize Model Improvements. In CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3544548.3580790
  154. Generating Phishing Attacks Using ChatGPT. arXiv 2305.05133 (2023). http://arxiv.org/abs/2305.05133
  155. Aequitas: A Bias and Fairness Audit Toolkit. arXiv 1811.05577 (2019). http://arxiv.org/abs/1811.05577
  156. Juan Pablo Sarmiento and Alyssa Friend Wise. 2022. Participatory and Co-Design of Learning Analytics: An Initial Review of the Literature. In LAK22: 12th International Learning Analytics and Knowledge Conference. https://doi.org/10.1145/3506860.3506910
  157. Shlomo S. Sawilowsky. 2009. New Effect Size Rules of Thumb. Journal of Modern Applied Statistical Methods 8 (2009). https://doi.org/10.22237/jmasm/1257035100
  158. Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLP. arXiv 2103.00453 (2021). http://arxiv.org/abs/2103.00453
  159. Principles to Practices for Responsible AI: Closing the Gap. arXiv 2006.04707 (2020). http://arxiv.org/abs/2006.04707
  160. Digital Nudging: Guiding Online User Choices through Interface Design. Commun. ACM 61 (2018). https://doi.org/10.1145/3213765
  161. Howard J Seltman. 2012. Experimental Design and Analysis.
  162. S. S. Shapiro and M. B. Wilk. 1965. An Analysis of Variance Test for Normality (Complete Samples). Biometrika 52 (1965). https://doi.org/10.1093/biomet/52.3-4.591
  163. Meaningful Context, a Red Flag, or Both? Preferences for Enhanced Misinformation Warnings Among US Twitter Users. In 2022 European Symposium on Usable Security. https://doi.org/10.1145/3549015.3555671
  164. Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction. In Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society. https://doi.org/10.1145/3600211.3604673
  165. Value Cards: An Educational Toolkit for Teaching Social Impacts of Machine Learning through Deliberation. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency. https://doi.org/10.1145/3442188.3445971
  166. Model Evaluation for Extreme Risks. arXiv 2305.15324 (2023). http://arxiv.org/abs/2305.15324
  167. Thinking through and Writing about Research Ethics beyond” Broader Impact”. arXiv preprint arXiv:2104.08205 (2021).
  168. Gabriel Simmons. 2023. Moral Mimicry: Large Language Models Produce Moral Rationalizations Tailored to Political Identity. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop). https://doi.org/10.18653/v1/2023.acl-srw.40
  169. Guy Simon. 2020. OpenWeb Tests the Impact of “Nudges” in Online Discussions. OpenWeb Blog (2020).
  170. TensorFlow.Js: Machine Learning for the Web and Beyond. arXiv (2019). https://arxiv.org/abs/1901.05350
  171. REAL ML: Recognizing, Exploring, and Articulating Limitations of Machine Learning Research. In 2022 ACM Conference on Fairness, Accountability, and Transparency. https://doi.org/10.1145/3531146.3533122
  172. Incorporating Ethics in Computing Courses: Barriers, Support, and Perspectives from Educators. In Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1. https://doi.org/10.1145/3545945.3569855
  173. Irene Solaiman and Christy Dennison. 2021. Process for Adapting Language Models to Society (PALMS) with Values-Targeted Datasets. In Advances in Neural Information Processing Systems, Vol. 34. https://proceedings.neurips.cc/paper_files/paper/2021/file/2e855f9489df0712b4bd8ea9e2848c5a-Paper.pdf
  174. Beyond the Imitation Game: Quantifying and Extrapolating the Capabilities of Language Models. arXiv preprint arXiv:2206.04615 (2022). http://arxiv.org/abs/2206.04615
  175. Consequences, Schmonsequences! Considering the Future as Part of Publication and Peer Review in Computing Research. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems.
  176. Harini Suresh and John Guttag. 2021. A Framework for Understanding Sources of Harm throughout the Machine Learning Life Cycle. In Equity and Access in Algorithms, Mechanisms, and Optimization. https://doi.org/10.1145/3465416.3483305
  177. Elham Tabassi. 2023. AI Risk Management Framework: AI RMF (1.0). Technical Report. National Institute of Standards and Technology. https://doi.org/10.6028/NIST.AI.100-1
  178. Gemini: A Family of Highly Capable Multimodal Models. arXiv preprint arXiv:2312.11805 (2023). https://arxiv.org/abs/2312.11805
  179. The Language Interpretability Tool: Extensible, Interactive Visualizations and Analysis for NLP Models. In EMNLP Demo. https://doi.org/10.18653/v1/2020.emnlp-demos.15
  180. Llama 2: Open Foundation and Fine-Tuned Chat Models. arXiv 2307.09288 (2023). https://arxiv.org/abs/2307.09288
  181. Rama Adithya Varanasi and Nitesh Goyal. 2023. ‘‘It Is Currently Hodgepodge”: Examining AI/ML Practitioners’ Challenges during Co-Production of Responsible AI Values. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems.
  182. Error Analysis of Statistical Machine Translation Output. In Proceedings of the Fifth International Conference on Language Resources and Evaluation (LREC’06). http://www.lrec-conf.org/proceedings/lrec2006/pdf/413_pdf.pdf
  183. Designing Responsible AI: Adaptations of UX Practice to Meet Responsible AI Challenges. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3544548.3581278
  184. Wordflow: Social Prompt Engineering for Large Language Models. arXiv 2401.14447 (2024). http://arxiv.org/abs/2401.14447
  185. StickyLand: Breaking the Linear Presentation of Computational Notebooks. In CHI Conference on Human Factors in Computing Systems Extended Abstracts. https://doi.org/10.1145/3491101.3519653
  186. WizMap: Scalable Interactive Visualization for Exploring Large Machine Learning Embeddings. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations). https://aclanthology.org/2023.acl-demo.50
  187. Interpretability, Then What? Editing Machine Learning Models to Reflect Human Knowledge and Values. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD ’22). https://doi.org/10.1145/3534678.3539074
  188. NOVA: A Practical Method for Creating Notebook-Ready Visual Analytics. arXiv:2205.03963 (2022). http://arxiv.org/abs/2205.03963
  189. Fairlearn: Assessing and Improving Fairness of AI Systems. arXiv 2303.16626 (2023). http://arxiv.org/abs/2303.16626
  190. Ethical and Social Risks of Harm from Language Models. arXiv 2112.04359 (2021). https://doi.org/10.48550/arXiv.2112.04359
  191. Sociotechnical Safety Evaluation of Generative AI Systems. arXiv 2310.11986 (2023). http://arxiv.org/abs/2310.11986
  192. Taxonomy of Risks Posed by Language Models. In 2022 ACM Conference on Fairness, Accountability, and Transparency. https://doi.org/10.1145/3531146.3533088
  193. Benjamin Weiser and Nate Schweber. 2023. The ChatGPT Lawyer Explains Himself. The New York Times (2023). https://www.nytimes.com/2023/06/08/nyregion/lawyer-chatgpt-sanctions.html
  194. Challenges in Detoxifying Language Models. In Findings of the Association for Computational Linguistics: EMNLP 2021. https://doi.org/10.18653/v1/2021.findings-emnlp.210
  195. The What-If Tool: Interactive Probing of Machine Learning Models. TVCG 26 (2019). https://doi.org/10.1109/TVCG.2019.2934619
  196. Meredith Whittaker. 2021. The Steep Cost of Capture. Interactions 28 (2021). https://doi.org/10.1145/3488666
  197. Richmond Y Wong and Vera Khovanskaya. 2018. Speculative Design in HCI: From Corporate Imaginations to Critical Orientations.
  198. Seeing Like a Toolkit: How Toolkits Envision the Work of AI Ethics. Proceedings of the ACM on Human-Computer Interaction 7 (2023). https://doi.org/10.1145/3579621
  199. Richmond Y Wong and Tonya Nguyen. 2021. Timelines: A World-Building Activity for Values Advocacy. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems.
  200. RECAST: Enabling User Recourse and Interpretability of Toxicity Detection Models with Interactive Visualization. Proceedings of the ACM on Human-Computer Interaction 5 (2021). https://doi.org/10.1145/3449280
  201. A Comparative Analysis of Industry Human-AI Interaction Guidelines. http://arxiv.org/abs/2010.11761
  202. Do Security Toolbars Actually Prevent Phishing Attacks?. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/1124772.1124863
  203. Errudite: Scalable, Reproducible, and Testable Error Analysis. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. https://doi.org/10.18653/v1/P19-1073
  204. AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Prompts. In CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3491102.3517582
  205. Detoxifying Language Models Risks Marginalizing Minority Voices. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. https://doi.org/10.18653/v1/2021.naacl-main.190
  206. Investigating How Practitioners Use Human-AI Guidelines: A Case Study on the People + AI Guidebook. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3544548.3580900
  207. Why Johnny Can’t Prompt: How Non-AI Experts Try (and Fail) to Design LLM Prompts. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3544548.3581388
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (5)
  1. Zijie J. Wang (39 papers)
  2. Chinmay Kulkarni (15 papers)
  3. Lauren Wilcox (10 papers)
  4. Michael Terry (25 papers)
  5. Michael Madaio (15 papers)
Citations (19)
View on Semantic Scholar

Summary

Farsight: Enhancing Responsible AI Practice in LLM Application Prototyping

Introduction to Farsight

Recent advancements in AI have put powerful tools like LLMs into the hands of a broader spectrum of developers, including those without specialized AI training. However, this democratization comes with the increased responsibility of identifying and mitigating potential harms that these technologies could cause. This paper introduces Farsight, an innovative in situ interactive tool designed to help AI prototypers envision potential harms associated with their LLM-powered applications during the early stages of prototyping.

Key Features of Farsight

Farsight incorporates several novel aspects that distinguish it from existing resources for responsible AI:

  • In Situ Integration: Farsight is designed to be a seamless part of AI prototyping environments, such as Google AI Studio and Jupyter Notebook. This design choice ensures that considerations of potential harms are naturally incorporated into the workflow of AI application development.
  • Interactive Harm Envisioning: Utilizing embedding similarities, Farsight dynamically suggests news articles about relevant AI incidents, thus providing contextually relevant examples of potential harms. It further leverages the capabilities of LLMs to generate and allow users to interactively explore potential use cases, stakeholders, and harms.
  • User Engagement and Agency: By enabling users to edit and augment LLM-generated content, Farsight ensures that AI prototypers engage critically with the tool, encouraging a deeper consideration of potential harms beyond the immediate functionalities of their application.
  • Open-source Implementation: To facilitate widespread adoption and future enhancements, Farsight is made available as an open-source tool. Its implementation as a collection of web components ensures compatibility with current web-based AI development environments.

Evaluation and Insights

The user paper involving 42 AI prototypers demonstrated Farsight's effectiveness in fostering responsible AI awareness. Key findings include:

  • Increased Harm Awareness: Participants who used Farsight were significantly better at identifying potential harms independently after the intervention, indicating that Farsight effectively enhances users' harm envisioning capabilities.
  • Shift in Focus: Qualitative feedback suggests that Farsight shifts users' attention from the AI model to its end-users, promoting a broader consideration of indirect stakeholders and cascading harms.
  • High Usability and Engagement: Farsight was rated favorably in terms of usability and usefulness compared to other resources. Participants found it to be a useful addition to their prototyping workflows, appreciating its seamless integration and the agency it provides.

Future Directions

While Farsight marks a significant step toward integrating responsible AI practices into the AI development lifecycle, it also opens avenues for further research. Future work could explore methods for incorporating actionable harm mitigation strategies within Farsight, addressing the current limitation of focusing primarily on harm identification. Such enhancements could offer AI prototypers not only the means to envision potential harms but also the tools to counteract them effectively from the early stages of AI application development.

Farsight's introduction into the AI prototyping process represents a substantive contribution toward fostering a culture of responsibility among AI developers. As AI technologies continue to evolve, tools like Farsight will be crucial in ensuring that these advancements proceed with due regard for their societal impact.

File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown
Github Logo Streamline Icon: https://streamlinehq.com

GitHub

  1. GitHub - PAIR-code/farsight: In situ interactive widgets for responsible AI 🌱 (25 stars)