Papers
Topics
Authors
Recent
Search
2000 character limit reached

Can Large Language Models Automatically Jailbreak GPT-4V?

Published 23 Jul 2024 in cs.CL | (2407.16686v2)

Abstract: GPT-4V has attracted considerable attention due to its extraordinary capacity for integrating and processing multimodal information. At the same time, its ability of face recognition raises new safety concerns of privacy leakage. Despite researchers' efforts in safety alignment through RLHF or preprocessing filters, vulnerabilities might still be exploited. In our study, we introduce AutoJailbreak, an innovative automatic jailbreak technique inspired by prompt optimization. We leverage LLMs for red-teaming to refine the jailbreak prompt and employ weak-to-strong in-context learning prompts to boost efficiency. Furthermore, we present an effective search method that incorporates early stopping to minimize optimization time and token expenditure. Our experiments demonstrate that AutoJailbreak significantly surpasses conventional methods, achieving an Attack Success Rate (ASR) exceeding 95.3\%. This research sheds light on strengthening GPT-4V security, underscoring the potential for LLMs to be exploited in compromising GPT-4V integrity.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (55)
  1. Touchstone: Evaluating vision-language models by language models.
  2. Training a helpful and harmless assistant with reinforcement learning from human feedback.
  3. Jailbreaking black box large language models in twenty queries. arXiv preprint arXiv:2310.08419.
  4. Deep reinforcement learning from human preferences.
  5. Multilingual jailbreak challenges in large language models. arXiv preprint arXiv:2310.06474.
  6. How robust is google’s bard to adversarial image attacks?
  7. Figstep: Jailbreaking large vision-language models via typographic visual prompts. arXiv preprint arXiv:2311.05608.
  8. Trustgpt: A benchmark for trustworthy and responsible large language models. arXiv preprint arXiv:2306.11507.
  9. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models.
  10. Deepinception: Hypnotize large language model to be jailbreaker.
  11. Evaluating object hallucination in large vision-language models.
  12. I think, therefore i am: Awareness in large language models.
  13. Goat-bench: Safety insights to large multimodal models through meme-based social abuse.
  14. Visual instruction tuning.
  15. Autodan: Generating stealthy jailbreak prompts on aligned large language models. arXiv preprint arXiv:2310.04451.
  16. Safety of multimodal large language models on images and text.
  17. Trustworthy llms: a survey and guideline for evaluating large language models’ alignment. arXiv preprint arXiv:2308.05374.
  18. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV).
  19. Umap: Uniform manifold approximation and projection for dimension reduction.
  20. OpenAI. 2023a. Gpt-4v(ision) system card.
  21. OpenAI. 2023b. Openai embedding models.
  22. OpenAI. 2023c. Openai moderation api. https://platform.openai.com/docs/guides/moderation.
  23. Training language models to follow instructions with human feedback.
  24. Emanuel Parzen. 1962. On estimation of a probability density function and mode. The annals of mathematical statistics, 33(3):1065–1076.
  25. Lutz Prechelt. 2002. Early stopping-but when? In Neural Networks: Tricks of the trade, pages 55–69. Springer.
  26. Visual adversarial examples jailbreak aligned large language models.
  27. Fine-tuning aligned language models compromises safety, even when users do not intend to!
  28. Smoothllm: Defending large language models against jailbreaking attacks. arXiv preprint arXiv:2310.03684.
  29. Towards an exhaustive evaluation of vision-language foundation models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 339–352.
  30. On second thought, let’s not think step by step! bias and toxicity in zero-shot reasoning.
  31. " do anything now": Characterizing and evaluating in-the-wild jailbreak prompts on large language models. arXiv preprint arXiv:2308.03825.
  32. "do anything now": Characterizing and evaluating in-the-wild jailbreak prompts on large language models.
  33. shujujishi. 2019. Face datasets of ten chinese stars.
  34. Why do universal adversarial attacks work on large language models?: Geometry might be the answer.
  35. Temporal insight enhancement: Mitigating temporal hallucination in multimodal large language models. arXiv preprint arXiv:2401.09861.
  36. Trustllm: Trustworthiness in large language models. arXiv preprint arXiv:2401.05561.
  37. VISHESH THAKUR. 2022. Celebrity face image dataset.
  38. Decodingtrust: A comprehensive assessment of trustworthiness in gpt models. arXiv preprint arXiv:2306.11698.
  39. Haoran Wang and Kai Shu. 2023. Backdoor activation attack: Attack large language models using activation steering for safety-alignment.
  40. Jailbroken: How does llm safety training fail? arXiv preprint arXiv:2307.02483.
  41. Jailbreak and guard aligned language models with only few in-context demonstrations. arXiv preprint arXiv:2310.06387.
  42. Defending chatgpt against jailbreak attack via self-reminder.
  43. Jailbreaking gpt-4v via self-adversarial attacks with system prompts.
  44. Lvlm-ehub: A comprehensive evaluation benchmark for large vision-language models. arXiv preprint arXiv:2306.09265.
  45. Large language models as optimizers.
  46. On early stopping in gradient descent learning. Constructive Approximation, 26:289–315.
  47. A survey on multimodal large language models.
  48. Woodpecker: Hallucination correction for multimodal large language models. arXiv preprint arXiv:2310.16045.
  49. Low-resource languages jailbreak gpt-4. arXiv preprint arXiv:2310.02446.
  50. Low-resource languages jailbreak gpt-4.
  51. Vision-language models for vision tasks: A survey.
  52. On evaluating adversarial robustness of large vision-language models. In Thirty-seventh Conference on Neural Information Processing Systems.
  53. Robust prompt optimization for defending language models against jailbreaking attacks.
  54. Safety fine-tuning at (almost) no cost: A baseline for vision large language models.
  55. Universal and transferable adversarial attacks on aligned language models.
Citations (1)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up

Tweets

Sign up for free to view the 2 tweets with 0 likes about this paper.

Sign Up for Free