SimpleSafetyTests: a Test Suite for Identifying Critical Safety Risks in Large Language Models (2311.08370v2)

Abstract: The past year has seen rapid acceleration in the development of LLMs. However, without proper steering and safeguards, LLMs will readily follow malicious instructions, provide unsafe advice, and generate toxic content. We introduce SimpleSafetyTests (SST) as a new test suite for rapidly and systematically identifying such critical safety risks. The test suite comprises 100 test prompts across five harm areas that LLMs, for the vast majority of applications, should refuse to comply with. We test 11 open-access and open-source LLMs and four closed-source LLMs, and find critical safety weaknesses. While some of the models do not give a single unsafe response, most give unsafe responses to more than 20% of the prompts, with over 50% unsafe responses in the extreme. Prepending a safety-emphasising system prompt substantially reduces the occurrence of unsafe responses, but does not completely stop them from happening. Trained annotators labelled every model response to SST (n = 3,000). We use these annotations to evaluate five AI safety filters (which assess whether a models' response is unsafe given a prompt) as a way of automatically evaluating models' performance on SST. The filters' performance varies considerably. There are also differences across the five harm areas, and on the unsafe versus safe responses. The widely-used Perspective API has 72% accuracy and a newly-created zero-shot prompt to OpenAI's GPT-4 performs best with 89% accuracy. Content Warning: This paper contains prompts and responses that relate to child abuse, suicide, self-harm and eating disorders, scams and fraud, illegal items, and physical harm.

Evaluating Safety Risks in LLMs via SIMPLESAFETYTESTS

The proliferation of LLMs has underscored the critical need for robust safety measures that prevent these models from generating harmful outputs. The paper "SIMPLESAFETYTESTS: A Test Suite for Identifying Critical Safety Risks in LLMs" addresses this necessity by providing a comprehensive test suite aimed at identifying safety weaknesses in these models. The authors present a structured approach to evaluating 15 LLMs using a curated set of prompts that span five harm categories: Suicide, Self-Harm, and Eating Disorders; Physical Harm; Illegal and Highly Regulated Items; Scams and Fraud; and Child Abuse.

Key Findings and Results

The SIMPLESAFETYTESTS (SST) test suite comprises 100 meticulously crafted prompts designed to elicit potentially unsafe responses from LLMs. The paper reports that, without a safety-emphasising system prompt, 20% of the total responses from LLMs were unsafe. Closed-source models demonstrated remarkably better performance, with only 2% of their responses being unsafe compared to 27% from open models. Notably, models like Claude 2.1 and Falcon (40B) yielded no unsafe responses, while others like Dolly v2 (12B) showed considerable safety deficiencies, with unsafe responses at a rate of 69%.

The introduction of a safety-emphasising system prompt reduced unsafe responses by an average of nine percentage points, illustrating its utility in enhancing model safety. However, this system prompt did not eliminate safety risks entirely, highlighting that the underlying issue may extend beyond simple inference-time interventions.

Implications and Future Directions

The implications of this research are significant for both theoretical advancement and practical deployment of LLMs. From a theoretical perspective, the paper suggests that LLM safety is inherently tied to context-specific evaluations. The testing suite's ability to identify safety risks provides a valuable tool for understanding how LLMs can be manipulated into producing harmful content, thereby guiding the development of more robust alignment techniques.

Practically, the findings underscore the urgency of deploying LLMs with additional layers of security. Developers and commercial labs are prompted to integrate system prompts that prioritize safety, potentially incorporating more advanced techniques like red-teaming and reinforcement learning from human feedback (RLHF) for heightened security. Moreover, the paper accentuates the disparity in safety compliance between open and closed-source models, perhaps indicating a need for increased transparency and collaboration in the open-source community to address these gaps.

Automated Evaluation of Safety

The research extends into exploring the feasibility of automating the evaluation of LLM responses using pre-existing safety filters. Despite variance in accuracy rates, the paper identifies OpenAI's zero-shot prompt to GPT-4 as the most effective tool, achieving 89% accuracy. This suggests potential for developing automated systems to monitor and evaluate LLM interactions, facilitating large-scale safety assessments.

Limitations and Prospects

While SIMPLESAFETYTESTS provide a clear-cut methodology for evaluating safety risks, they are limited by breadth, covering English test prompts and a fixed set of harm categories. Future expansions could encompass multilingual prompts and additional harm areas to increase relevancy across diverse applications. Moreover, the paper advocates for perturbation-based testing to enhance robustness, a promising avenue for further exploration.

Overall, this paper lays foundational groundwork for the systematic evaluation of LLM safety. It calls for strengthened safety measures that reflect the nuanced challenges posed by the rapid integration of LLMs into various applications. The structured methodology and preliminary findings serve as a roadmap for future research endeavors aimed at ensuring the safe and responsible development of AI technologies.