It is Time to Develop an Auditing Framework to Promote Value Aware Chatbots (2409.01539v1)
Abstract: The launch of ChatGPT in November 2022 marked the beginning of a new era in AI, the availability of generative AI tools for everyone to use. ChatGPT and other similar chatbots boast a wide range of capabilities from answering student homework questions to creating music and art. Given the large amounts of human data chatbots are built on, it is inevitable that they will inherit human errors and biases. These biases have the potential to inflict significant harm or increase inequity on different subpopulations. Because chatbots do not have an inherent understanding of societal values, they may create new content that is contrary to established norms. Examples of concerning generated content includes child pornography, inaccurate facts, and discriminatory posts. In this position paper, we argue that the speed of advancement of this technology requires us, as computer and data scientists, to mobilize and develop a values-based auditing framework containing a community established standard set of measurements to monitor the health of different chatbots and LLMs. To support our argument, we use a simple audit template to share the results of basic audits we conduct that are focused on measuring potential bias in search engine style tasks, code generation, and story generation. We identify responses from GPT 3.5 and GPT 4 that are both consistent and not consistent with values derived from existing law. While the findings come as no surprise, they do underscore the urgency of developing a robust auditing framework for openly sharing results in a consistent way so that mitigation strategies can be developed by the academic community, government agencies, and companies when our values are not being adhered to. We conclude this paper with recommendations for value-based strategies for improving the technologies.
- Persistent anti-muslim bias in large language models. In Conference on AI, Ethics, and Society, pages 298–306.
- Storium: A dataset and evaluation platform for machine-in-the-loop story generation. arXiv preprint arXiv:2010.01717.
- Racial disparity in natural language processing: A case study of social media african-american english. arXiv:1707.00061.
- BLS (2023a). Occupational outlook handbook. https://www.bls.gov/ooh/.
- BLS (2023b). Occupational outlook handbook: Software developers, quality assurance analysts, and testers. https://www.bls.gov/ooh/computer-and-information-technology/software-developers.htm.
- BLS (2024). Labor force statistics from the current population survey. https://www.bls.gov/cps/cpsaat11.htm.
- Large language models in machine translation.
- Generative ai could raise global gdp by 7%.
- Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901.
- Carbonaro, G. (2024). Openai’s chatgpt chatbot tops the list but these are the 9 other most popular ai tools just now.
- Medically aware gpt-3 as a data generator for medical dialogue summarization. In Machine Learning for Healthcare Conference, pages 354–372. PMLR.
- Creative writing with a machine in the loop: Case studies on slogans and stories. In Conference on Intelligent User Interfaces, pages 329–340.
- Costa-jussà, M. R. (2019). An analysis of gender bias studies in natural language processing. Nature Machine Intelligence, 1(11):495–496.
- Can gpt-3 pass a writer’s turing test? Journal of Cultural Analytics, 5(2).
- FDA (2018). Software as a medical device (samd). https://www.fda.gov/medical-devices/digital-health-center-excellence/software-medical-device-samd.
- Future of Life Institute (2023). Pause giant ai experiments: An open letter.
- Chatgpt outperforms crowd-workers for text-annotation tasks. arXiv:2303.15056.
- Glassdoor (2023a). Discover careers. https://www.glassdoor.com/Career/index.htm.
- Glassdoor (2023b). Software developer overview united states. https://www.glassdoor.com/Career/software-developer-career_KO0,18.htm.
- The times sues openai and microsoft over a.i. use of copyrighted work.
- Evaluating large language models in generating synthetic hci research data: a case study. In Conference on Human Factors in Computing Systems, pages 1–19.
- Dual learning for machine translation. Advances in neural information processing systems, 29.
- IBM (2024). Foundation models: Opportunities, risks and mitigations.
- Auditing the ai auditors: A framework for evaluating fairness and bias in high stakes ai predictive models. American Psychologist, 78(1):36.
- A normative account of confirmation bias during reinforcement learning. Neural Computation, 34(2):307–337.
- Gender and representation bias in gpt-3 generated stories. In Workshop on Narrative Understanding, pages 48–55.
- Mayson, S. G. (2019). Bias in, bias out. The Yale Law Journal, 128(8):2218–2300.
- Combining the language model and inference network approaches to retrieval. Information processing & management, 40(5):735–750.
- Do we still need human assessors? prompt-based gpt-3 user simulation in conversational ai. In Conference on Conversational User Interfaces, pages 1–6.
- Nakatani, T. (2019). Improving transformer-based end-to-end speech recognition with connectionist temporal classification and language model integration. In Proc. Interspeech, volume 2019.
- Confirmation bias in human reinforcement learning: Evidence from counterfactual feedback processing. PLoS computational biology, 13(8):e1005684.
- Closing the ai accountability gap: Defining an end-to-end framework for internal algorithmic auditing. In Conference on fairness, accountability, and transparency, pages 33–44.
- Social bias frames: Reasoning about social and power implications of language. arXiv:1911.03891.
- Schechner, S. (2023). Chatgpt and advanced ai face new regulatory push in europe.
- Predictive biases in natural language processing models: A conceptual framework and overview. arXiv:1912.11078.
- Component fusion: Learning replaceable language model component for end-to-end speech recognition system. In Conference on Acoustics, Speech, and Signal Processing, pages 5361–5635. IEEE.
- Us begins study of possible rules to regulate ai like chatgpt.
- Indri: A language model-based search engine for complex queries. In Conference on intelligent analysis, volume 2, pages 2–6.
- Confirmation bias optimizes reward learning. BioRxiv, pages 2021–02.
- A comparison of techniques for language model integration in encoder-decoder speech recognition. In IEEE spoken language technology workshop (SLT), pages 369–375. IEEE.
- Want to reduce labeling cost? gpt-3 can help. arXiv:2108.13487.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.
