BSPA: Exploring Black-box Stealthy Prompt Attacks against Image Generators (2402.15218v1)
Abstract: Extremely large image generators offer significant transformative potential across diverse sectors. It allows users to design specific prompts to generate realistic images through some black-box APIs. However, some studies reveal that image generators are notably susceptible to attacks and generate Not Suitable For Work (NSFW) contents by manually designed toxin texts, especially imperceptible to human observers. We urgently need a multitude of universal and transferable prompts to improve the safety of image generators, especially black-box-released APIs. Nevertheless, they are constrained by labor-intensive design processes and heavily reliant on the quality of the given instructions. To achieve this, we introduce a black-box stealthy prompt attack (BSPA) that adopts a retriever to simulate attacks from API users. It can effectively harness filter scores to tune the retrieval space of sensitive words for matching the input prompts, thereby crafting stealthy prompts tailored for image generators. Significantly, this approach is model-agnostic and requires no internal access to the model's features, ensuring its applicability to a wide range of image generators. Building on BSPA, we have constructed an automated prompt tool and a comprehensive prompt attack dataset (NSFWeval). Extensive experiments demonstrate that BSPA effectively explores the security vulnerabilities in a variety of state-of-the-art available black-box models, including Stable Diffusion XL, Midjourney, and DALL-E 2/3. Furthermore, we develop a resilient text filter and offer targeted recommendations to ensure the security of image generators against prompt attacks in the future.
- Square attack: a query-efficient black-box adversarial attack via random search. In European conference on computer vision, pages 484–501. Springer.
- Improving black-box adversarial attacks with a transfer-based prior. Advances in neural information processing systems, 32.
- Introduction to derivative-free optimization. SIAM.
- Dall·e mini.
- Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805.
- Black-box prompt learning for pre-trained language models. Transactions on Machine Learning Research.
- Laura Hanu and Unitary team. 2020. Detoxify. Github. https://github.com/unitaryai/detoxify.
- Black-box adversarial attacks with limited queries and information. In International conference on machine learning, pages 2137–2146. PMLR.
- Dall-e-bot: Introducing web-scale diffusion models to robotics. IEEE Robotics and Automation Letters.
- Dense passage retrieval for open-domain question answering. arXiv preprint arXiv:2004.04906.
- Towards safe self-distillation of internet-scale text-to-image diffusion models. arXiv preprint arXiv:2307.05977.
- Microsoft coco: Common objects in context. In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, pages 740–755. Springer.
- Black-box few-shot knowledge distillation. In European Conference on Computer Vision, pages 196–211. Springer.
- Unsafe diffusion: On the generation of unsafe images and hateful memes from text-to-image models. ACM SIGSAC Conference on Computer and Communications Security.
- Zero-shot text-to-image generation. In International Conference on Machine Learning, pages 8821–8831. PMLR.
- Red-teaming the stable diffusion safety filter. arXiv preprint arXiv:2210.04610.
- Luis Miguel Rios and Nikolaos V Sahinidis. 2013. Derivative-free optimization: a review of algorithms and comparison of software implementations. Journal of Global Optimization, 56:1247–1293.
- Beyond the ml model: Applying safety engineering frameworks to text-to-image development. In Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society, pages 70–83.
- The probabilistic relevance framework: Bm25 and beyond. Foundations and Trends® in Information Retrieval, 3(4):333–389.
- High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695.
- Photorealistic text-to-image diffusion models with deep language understanding. Advances in Neural Information Processing Systems, 35:36479–36494.
- Safe latent diffusion: Mitigating inappropriate degeneration in diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22522–22531.
- Laion-5b: An open large-scale dataset for training next generation image-text models. Advances in Neural Information Processing Systems, 35:25278–25294.
- Laion-400m: Open dataset of clip-filtered 400 million image-text pairs. arXiv preprint arXiv:2111.02114.
- Solving inverse problems in medical imaging with score-based generative models. In International Conference on Learning Representations.
- Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations.
- Black-box tuning for language-model-as-a-service. In Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pages 20841–20855. PMLR.
- Yfcc100m: The new data in multimedia research. Communications of the ACM, 59(2):64–73.
- Evil geniuses: Delving into the safety of llm-based agents. arXiv preprint arXiv:2311.11855.
- Zi Wang. 2021. Zero-shot knowledge distillation from a decision-based black-box model. In International Conference on Machine Learning, pages 10675–10685. PMLR.
- Finetuned language models are zero-shot learners. arXiv preprint arXiv:2109.01652.
- Judging llm-as-a-judge with mt-bench and chatbot arena. arXiv preprint arXiv:2306.05685.
- Yu Tian (249 papers)
- Xiao Yang (158 papers)
- Yinpeng Dong (102 papers)
- Heming Yang (3 papers)
- Hang Su (224 papers)
- Jun Zhu (424 papers)