An AI System Evaluation Framework for Advancing AI Safety: Terminology, Taxonomy, Lifecycle Mapping (2404.05388v3)
Abstract: The advent of advanced AI underscores the urgent need for comprehensive safety evaluations, necessitating collaboration across communities (i.e., AI, software engineering, and governance). However, divergent practices and terminologies across these communities, combined with the complexity of AI systems-of which models are only a part-and environmental affordances (e.g., access to tools), obstruct effective communication and comprehensive evaluation. This paper proposes a framework for AI system evaluation comprising three components: 1) harmonised terminology to facilitate communication across communities involved in AI safety evaluation; 2) a taxonomy identifying essential elements for AI system evaluation; 3) a mapping between AI lifecycle, stakeholders, and requisite evaluations for accountable AI supply chain. This framework catalyses a deeper discourse on AI system evaluation beyond model-centric approaches.
- AI Safety Institute. Ai safety institute approach to evaluations. https://www.gov.uk/government/publications/ai-safety-institute-approach-to-evaluations/ai-safety-institute-approach-to-evaluations, 2024.
- Managing ai risks in an era of rapid progress. arXiv preprint arXiv:2310.17688, 2023.
- Responsible AI: Best Practices for Creating Trustworthy AI Systems. Pearson Education, 2023.
- Responsible ai pattern catalogue: A collection of best practices for ai governance and engineering. ACM Computing Surveys, 2023.
- The White House. Executive order on the safe, secure, and trustworthy development and use of artificial intelligence. https://www.whitehouse.gov/briefing-room/presidential-actions/2023/10/30/executive-order-on-the-safe-secure-and-trustworthy-development-and-use-of-artificial-intelligence/, 2023.
- The bletchley declaration by countries attending the ai safety summit, 1-2 november 2023. https://www.gov.uk/government/publications/ai-safety-summit-2023-the-bletchley-declaration/the-bletchley-declaration-by-countries-attending-the-ai-safety-summit-1-2-november-2023, 2023.
- Decodingtrust: A comprehensive assessment of trustworthiness in gpt models. arXiv preprint arXiv:2306.11698, 2023.
- Don’t make your llm an evaluation benchmark cheater. arXiv preprint arXiv:2311.01964, 2023.
- Selfcheckgpt: Zero-resource black-box hallucination detection for generative large language models. arXiv preprint arXiv:2303.08896, 2023.
- Trustllm: Trustworthiness in large language models. arXiv preprint arXiv:2401.05561, 2024.
- The shift from models to compound ai systems. https://bair.berkeley.edu/blog/2024/02/18/compound-ai-systems/, 2024.
- Evaluating llm systems: Metrics, challenges, and best practices. https://medium.com/data-science-at-microsoft/evaluating-llm-systems-metrics-challenges-and-best-practices-664ac25be7e5, 2024.
- A causal framework for ai regulation and auditing. 2024.
- AI Risk Management Framework (AI RMF 1.0), 2023.
- IEEE Computer Society. Guide to the software engineering body of knowledge (swebok): Version 3.0. Technical report, IEEE Computer Society, 2014.
- Sociotechnical safety evaluation of generative ai systems. arXiv preprint arXiv:2310.11986, 2023.
- Fairness testing: A comprehensive survey and analysis of trends. ACM Transactions on Software Engineering and Methodology, 2023.
- Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the Protection of Natural Persons with regard to the Processing of Personal Data and on the Free Movement of Such Data, and Repealing Directive 95/46/EC (General Data Protection Regulation). https://eur-lex.europa.eu/eli/reg/2016/679/oj, 2016.
- Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311–318, 2002.
- Navigating privacy and copyright challenges across the data lifecycle of generative ai. arXiv preprint arXiv:2311.18252, 2023.
- Glue: A multi-task benchmark and analysis platform for natural language understanding. arXiv preprint arXiv:1804.07461, 2018.
- Shieldlm: Empowering llms as aligned, customizable and explainable safety detectors. arXiv preprint arXiv:2402.16444, 2024.
- Towards a responsible ai metrics catalogue: A collection of metrics for ai accountability. In 3rd International Conference on AI Engineering–Software Engineering for AI (CAIN ’24), 2024.
- Boming Xia (14 papers)
- Qinghua Lu (100 papers)
- Liming Zhu (101 papers)
- Zhenchang Xing (99 papers)