Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
41 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
41 tokens/sec
o3 Pro
7 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Safety Alignment for Vision Language Models (2405.13581v1)

Published 22 May 2024 in cs.CV and cs.AI

Abstract: Benefiting from the powerful capabilities of LLMs, pre-trained visual encoder models connected to an LLMs can realize Vision LLMs (VLMs). However, existing research shows that the visual modality of VLMs is vulnerable, with attackers easily bypassing LLMs' safety alignment through visual modality features to launch attacks. To address this issue, we enhance the existing VLMs' visual modality safety alignment by adding safety modules, including a safety projector, safety tokens, and a safety head, through a two-stage training process, effectively improving the model's defense against risky images. For example, building upon the LLaVA-v1.5 model, we achieve a safety score of 8.26, surpassing the GPT-4V on the Red Teaming Visual LLMs (RTVLM) benchmark. Our method boasts ease of use, high flexibility, and strong controllability, and it enhances safety while having minimal impact on the model's general performance. Moreover, our alignment strategy also uncovers some possible risky content within commonly used open-source multimodal datasets. Our code will be open sourced after the anonymous review.

PDF HTML Abstract
Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown Bookmark Chat Bubble Oval Streamline Icon: https://streamlinehq.com Chat (Pro)
Definition Search Book Streamline Icon: https://streamlinehq.com
References (49)
  1. Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond. arXiv preprint arXiv:2308.12966, 2023.
  2. Image hijacks: Adversarial images can control generative models at runtime, 2023.
  3. Introducing our multimodal models, 2023. URL https://www.adept.ai/blog/fuyu-8b.
  4. Image safeguarding: Reasoning with conditional vision language model and obfuscating unsafe content counterfactually, 2024.
  5. Honeybee: Locality-enhanced projector for multimodal llm. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024.
  6. Antifakeprompt: Prompt-tuned vision-language models are fake image detectors, 2023.
  7. Sharegpt4v: Improving large multi-modal models with better captions. arXiv preprint arXiv:2311.12793, 2023.
  8. Microsoft coco captions: Data collection and evaluation server. arXiv preprint arXiv:1504.00325, 2015.
  9. Cogview: Mastering text-to-image generation via transformers. Advances in Neural Information Processing Systems, 34:19822–19835, 2021.
  10. Internlm-xcomposer2: Mastering free-form text-image composition and comprehension in vision-language large model. arXiv preprint arXiv:2401.16420, 2024.
  11. Glm: General language model pretraining with autoregressive blank infilling. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp.  320–335, 2022.
  12. Mme: A comprehensive evaluation benchmark for multimodal large language models, 2024.
  13. Inducing high energy-latency of large vision-language models with verbose images, 2024.
  14. Figstep: Jailbreaking large vision-language models via typographic visual prompts, 2023.
  15. Onellm: One framework to align all modalities with language. arXiv preprint arXiv:2312.03700, 2023.
  16. Lora: Low-rank adaptation of large language models, 2021.
  17. Kim, A. Nsfw data scraper. https://github.com/alex000kim/nsfw_data_scraper, 2021.
  18. A hierarchical approach for generating descriptive image paragraphs, 2017.
  19. Seed-bench-2: Benchmarking multimodal large language models. arXiv preprint arXiv:2311.17092, 2023a.
  20. Seed-bench: Benchmarking multimodal llms with generative comprehension. arXiv preprint arXiv:2307.16125, 2023b.
  21. Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In International conference on machine learning, pp.  12888–12900. PMLR, 2022.
  22. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In International conference on machine learning, pp.  19730–19742. PMLR, 2023c.
  23. Red teaming visual language models, 2024.
  24. Vl-trojan: Multimodal instruction backdoor attacks against autoregressive visual language models, 2024.
  25. Microsoft coco: Common objects in context. In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, pp.  740–755. Springer, 2014.
  26. Improved baselines with visual instruction tuning, 2023a.
  27. Visual instruction tuning, 2023b.
  28. Llava-next: Improved reasoning, ocr, and world knowledge, January 2024a. URL https://llava-vl.github.io/blog/2024-01-30-llava-next/.
  29. Visual instruction tuning. Advances in neural information processing systems, 36, 2024b.
  30. Mm-safetybench: A benchmark for safety evaluation of multimodal large language models, 2024c.
  31. Mmbench: Is your multi-modal model an all-around player?, 2023c.
  32. Deep learning face attributes in the wild, 2015.
  33. Stable bias: Evaluating societal representations in diffusion models. In Oh, A., Neumann, T., Globerson, A., Saenko, K., Hardt, M., and Levine, S. (eds.), Advances in Neural Information Processing Systems, volume 36, pp.  56338–56351. Curran Associates, Inc., 2023. URL https://proceedings.neurips.cc/paper_files/paper/2023/file/b01153e7112b347d8ed54f317840d8af-Paper-Datasets_and_Benchmarks.pdf.
  34. OpenAI. Gpt-4 technical report, 2024.
  35. QResearch. llama3-vision-alpha. https://huggingface.co/qresearch/llama-3-vision-alpha, 2024.
  36. Learning transferable visual models from natural language supervision, 2021.
  37. Xstest: A test suite for identifying exaggerated safety behaviours in large language models, 2024.
  38. How many unicorns are in this image? a safety evaluation benchmark for vision llms, 2023.
  39. Towards understanding and detecting cyberbullying in real-world images. In Proceedings of the 28th Annual Network and Distributed System Security Symposium. Internet Society, 2021.
  40. Knowledge mining with scene text for fine-grained recognition, 2022.
  41. Jailbroken: How does llm safety training fail?, 2023.
  42. Adversarial prompt tuning for vision-language models, 2023a.
  43. A mutation-based method for multi-modal jailbreaking attack detection, 2023b.
  44. Meta-transformer: A unified framework for multimodal learning. arXiv preprint arXiv:2307.10802, 2023c.
  45. Privacyalert: A dataset for image privacy prediction. Proceedings of the International AAAI Conference on Web and Social Media, 16(1):1352–1361, May 2022. doi: 10.1609/icwsm.v16i1.19387. URL https://ojs.aaai.org/index.php/ICWSM/article/view/19387.
  46. On evaluating adversarial robustness of large vision-language models, 2023.
  47. Mquake: Assessing knowledge editing in language models via multi-hop questions, 2023.
  48. Safety fine-tuning at (almost) no cost: A baseline for vision large language models, 2024.
  49. Universal and transferable adversarial attacks on aligned language models, 2023.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (8)
  1. Zhendong Liu (16 papers)
  2. Yuanbi Nie (2 papers)
  3. Yingshui Tan (23 papers)
  4. Xiangyu Yue (93 papers)
  5. Qiushi Cui (6 papers)
  6. Chongjun Wang (27 papers)
  7. Xiaoyong Zhu (12 papers)
  8. Bo Zheng (205 papers)
Citations (3)
View on Semantic Scholar
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets