Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
38 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
41 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

GuardT2I: Defending Text-to-Image Models from Adversarial Prompts (2403.01446v2)

Published 3 Mar 2024 in cs.CV

Abstract: Recent advancements in Text-to-Image (T2I) models have raised significant safety concerns about their potential misuse for generating inappropriate or Not-Safe-For-Work (NSFW) contents, despite existing countermeasures such as NSFW classifiers or model fine-tuning for inappropriate concept removal. Addressing this challenge, our study unveils GuardT2I, a novel moderation framework that adopts a generative approach to enhance T2I models' robustness against adversarial prompts. Instead of making a binary classification, GuardT2I utilizes a LLM to conditionally transform text guidance embeddings within the T2I models into natural language for effective adversarial prompt detection, without compromising the models' inherent performance. Our extensive experiments reveal that GuardT2I outperforms leading commercial solutions like OpenAI-Moderation and Microsoft Azure Moderator by a significant margin across diverse adversarial scenarios. Our framework is available at https://github.com/cure-lab/GuardT2I.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (66)
  1. Lexica. https://lexica.art/, 2023.
  2. Stable Diffusion V1.5 checkpoint. https://huggingface.co/runwayml/stable-diffusion-v1-5?text=chi+venezuela+drogenius, 2023.
  3. Microsoft Azure Content Moderator. https://learn.microsoft.com/zh-cn/azure/ai-services/content-moderator/api-reference, 2024.
  4. AWS Comprehend. https://docs.aws.amazon.com/comprehend/latest/dg/what-is.html, 2024.
  5. SurrogatePrompt: Bypassing the Safety Filter of Text-To-Image Models via Substitution. arXiv preprint arXiv:2309.14122, 2023.
  6. Neural Machine Translation by Jointly Learning to Align and Translate. In Proceedings of the International Conference on Learning Representations, 2015.
  7. Scheduled sampling for sequence prediction with recurrent neural networks. Advances in neural information processing systems, 2015.
  8. Improving image generation with better captions. https://cdn.openai.com/papers/dall-e-3.pdf, 2023.
  9. Chicco, D. Siamese neural networks: An overview. Artificial neural networks, 2021.
  10. Supervised learning of universal sentence representations from natural language inference data. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp.  670–680, Copenhagen, Denmark, September 2017. Association for Computational Linguistics. doi: 10.18653/v1/D17-1070.
  11. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018.
  12. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the Annual Conference of the North American Chapter of the Association for Computational Linguistics, pp.  4171–4186, 2019.
  13. GLM: General Language Model Pretraining with Autoregressive Blank Infilling. 2022.
  14. Beam search strategies for neural machine translation. In Proceedings of the First Workshop on Neural Machine Translation, pp.  56–60, 2017.
  15. Erasing Concepts from Diffusion Models. arXiv preprint arXiv:2303.07345, 2023.
  16. Detoxify. https://github.com/unitaryai/detoxify, 2020.
  17. A Twofold Siamese Network for Real-Time Object Tracking. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.  4834–4843, 2018.
  18. HuggingFace. Safety Checker nested in Stable Diffusion. https://huggingface.co/CompVis/stable-diffusion-safety-checker, 2023.
  19. A survey on contrastive self-supervised learning. Technologies, 2020.
  20. Siamese neural networks for one-shot image recognition. In Proceedings of the International Conference on Machine Learning deep learning workshop, 2015.
  21. Ablating Concepts in Text-to-Image Diffusion Models. arXiv preprint arXiv:2303.13516, 2023.
  22. Holistic Evaluation of Text-To-Image Models. arXiv preprint arXiv:2311.04287, 2023.
  23. Leonardo.Ai. Leonardo.Ai. https://leonardo.ai/, 2023.
  24. Mixed Cross Entropy Loss for Neural Machine Translation. In Proceedings of the International Conference on Machine Learning, pp.  6425–6436, 2021.
  25. A holistic approach to undesired content detection in the real world. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pp.  15009–15018, 2023.
  26. Learned in translation: Contextualized word vectors. Advances in neural information processing systems, 30, 2017.
  27. Sdedit: Guided image synthesis and editing with stochastic differential equations. In International Conference on Learning Representations, 2021.
  28. Michellejieli. NSFW text classifier. https://huggingface.co/michellejieli/NSFW_text_classifier, 2023.
  29. Midjounery. Midjourney. https://midjourney.com/, 2023.
  30. Pytorch metric learning, 2020.
  31. GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models. In Proceedings of the International Conference on Machine Learning, pp.  16784–16804, 2022.
  32. OpenAI. Moderation overview. https://platform.openai.com/docs/guides/moderation/overview, 2023.
  33. PlaygroundAI. Playground. https://playgroundai.com/, 2023.
  34. SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis. arXiv preprint arXiv:2307.01952, 2023.
  35. Unsafe Diffusion: On the Generation of Unsafe Images and Hateful Memes From Text-To-Image Models. arXiv preprint arXiv:2305.13873, 2023.
  36. Improving language understanding by generative pre-training. OpenAI.
  37. Language models are unsupervised multitask learners. OpenAI, 2019.
  38. Learning Transferable Visual Models From Natural Language Supervision. In Proceedings of the International Conference on Machine Learning, pp.  8748–8763, 2021.
  39. Hierarchical Text-Conditional Image Generation with CLIP Latents. arXiv preprint arXiv:2204.06125, 2022.
  40. Red-Teaming the Stable Diffusion Safety Filter. arXiv preprint arXiv:2210.04610, 2022a.
  41. Red-Teaming the Stable Diffusion Safety Filter. arXiv preprint arXiv:2210.04610, 2022b.
  42. On the convergence of adam and beyond. In International Conference on Learning Representations, 2018.
  43. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2019.
  44. Making monolingual sentence embeddings multilingual using knowledge distillation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2020.
  45. High-Resolution Image Synthesis with Latent Diffusion Models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.  10674–10685, 2022.
  46. Leveraging pre-trained checkpoints for sequence generation tasks. Transactions of the Association for Computational Linguistics, 8:264–280, 2020.
  47. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.  22500–22510, 2023.
  48. Photorealistic text-to-image diffusion models with deep language understanding. Advances in Neural Information Processing Systems, 35:36479–36494, 2022a.
  49. Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding. In Proceedings of the Advances in Neural Information Processing Systems, 2022b.
  50. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108, 2019.
  51. Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.  22522–22531, 2023.
  52. LAION-5B: An Open Large-scale Dataset for Training Next Generation Image-text Models. In Proceedings of the Advances in Neural Information Processing Systems, 2022a.
  53. LAION-COCO. https://laion.ai/blog/laion-coco/, 2022b.
  54. Augmented sbert: Data augmentation method for improving bi-encoders for pairwise sentence scoring tasks. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2021a.
  55. Beir: A heterogenous benchmark for zero-shot evaluation of information retrieval models. In ”Proceedings of the Advances in Neural Information Processing Systems”, 2021b.
  56. Llama: open and efficient foundation language models. 2023a.
  57. Llama 2: Open foundation and fine-tuned chat models. 2023b.
  58. Ring-A-Bell! How Reliable are Concept Removal Methods for Diffusion Models? arXiv preprint arXiv:2310.10012, 2023.
  59. Attention is All you Need. In Proceedings of the Advances in Neural Information Processing Systems, pp.  5998–6008, 2017.
  60. Tsdae: Using transformer-based sequential denoising auto-encoderfor unsupervised sentence embedding learning. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2021.
  61. A learning algorithm for continually running fully recurrent neural networks. Neural computation, 1989.
  62. MMA-Diffusion: MultiModal Attack on Diffusion Models. arXiv preprint arXiv:2311.17516, 2023.
  63. Sneakyprompt: Jailbreaking text-to-image generative models. In Proceedings of the IEEE Symposium on Security and Privacy, 2024.
  64. To generate or not? safety-driven unlearned diffusion models are still easy to generate unsafe images… for now. arXiv preprint arXiv:2310.11868, 2023a.
  65. To generate or not? safety-driven unlearned diffusion models are still easy to generate unsafe images… for now. arXiv preprint arXiv:2310.11868, 2023b.
  66. Deeper and wider siamese networks for real-time visual tracking. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (5)
  1. Yijun Yang (46 papers)
  2. Ruiyuan Gao (18 papers)
  3. Xiao Yang (158 papers)
  4. Jianyuan Zhong (13 papers)
  5. Qiang Xu (129 papers)
Citations (8)
View on Semantic Scholar