Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
91 tokens/sec
Gemini 2.5 Pro Premium
52 tokens/sec
GPT-5 Medium
24 tokens/sec
GPT-5 High Premium
28 tokens/sec
GPT-4o
85 tokens/sec
DeepSeek R1 via Azure Premium
87 tokens/sec
GPT OSS 120B via Groq Premium
478 tokens/sec
Kimi K2 via Groq Premium
221 tokens/sec
2000 character limit reached

Enhancing Adversarial Attacks through Chain of Thought (2410.21791v1)

Published 29 Oct 2024 in cs.CL

Abstract: LLMs have demonstrated impressive performance across various domains but remain susceptible to safety concerns. Prior research indicates that gradient-based adversarial attacks are particularly effective against aligned LLMs and the chain of thought (CoT) prompting can elicit desired answers through step-by-step reasoning. This paper proposes enhancing the robustness of adversarial attacks on aligned LLMs by integrating CoT prompts with the greedy coordinate gradient (GCG) technique. Using CoT triggers instead of affirmative targets stimulates the reasoning abilities of backend LLMs, thereby improving the transferability and universality of adversarial attacks. We conducted an ablation study comparing our CoT-GCG approach with Amazon Web Services auto-cot. Results revealed our approach outperformed both the baseline GCG attack and CoT prompting. Additionally, we used Llama Guard to evaluate potentially harmful interactions, providing a more objective risk assessment of entire conversations compared to matching outputs to rejection phrases. The code of this paper is available at https://github.com/sujingbo0217/CS222W24-LLM-Attack.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (30)
  1. Gpt-4 technical report. arXiv preprint arXiv:2303.08774.
  2. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862.
  3. Improving image generation with better captions. Computer Science. https://cdn. openai. com/papers/dall-e-3. pdf, 2(3):8.
  4. On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258.
  5. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901.
  6. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality.
  7. Palm: Scaling language modeling with pathways. arxiv 2022. arXiv preprint arXiv:2204.02311, 10.
  8. Not what you’ve signed up for: Compromising real-world llm-integrated applications with indirect prompt injection. In Proceedings of the 16th ACM Workshop on Artificial Intelligence and Security, pages 79–90.
  9. Gradient-based adversarial attacks against text transformers. arXiv preprint arXiv:2104.13733.
  10. Catastrophic jailbreak of open-source llms via exploiting generation. arXiv preprint arXiv:2310.06987.
  11. Llama guard: Llm-based input-output safeguard for human-ai conversations. arXiv preprint arXiv:2312.06674.
  12. Mistral 7b. arXiv preprint arXiv:2310.06825.
  13. Large language models are zero-shot reasoners. Advances in neural information processing systems, 35:22199–22213.
  14. Toxicchat: Unveiling hidden challenges of toxicity detection in real-world user-ai conversation. arXiv preprint arXiv:2310.17389.
  15. Textattack: A framework for adversarial attacks, data augmentation, and adversarial training in nlp. arXiv preprint arXiv:2005.05909.
  16. Red teaming language models with language models. arXiv preprint arXiv:2202.03286.
  17. Subhro Roy and Dan Roth. 2016. Solving general arithmetic word problems. arXiv preprint arXiv:1608.01413.
  18. Autoprompt: Eliciting knowledge from language models with automatically generated prompts. arXiv preprint arXiv:2010.15980.
  19. Commonsenseqa: A question answering challenge targeting commonsense knowledge. arXiv preprint arXiv:1811.00937.
  20. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805.
  21. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971.
  22. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288.
  23. Jailbroken: How does llm safety training fail? Advances in Neural Information Processing Systems, 36.
  24. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837.
  25. Lilian Weng. 2023. Adversarial attacks on llms. lilianweng.github.io.
  26. Badchain: Backdoor chain-of-thought prompting for large language models. arXiv preprint arXiv:2401.12242.
  27. Gptfuzzer: Red teaming large language models with auto-generated jailbreak prompts. arXiv preprint arXiv:2309.10253.
  28. Opt: Open pre-trained transformer language models. arXiv preprint arXiv:2205.01068.
  29. Automatic chain of thought prompting in large language models. arXiv preprint arXiv:2210.03493.
  30. Universal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307.15043.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Dice Question Streamline Icon: https://streamlinehq.com

Follow-up Questions

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Authors (1)

  1. Jingbo Su
Github Logo Streamline Icon: https://streamlinehq.com

GitHub

  1. GitHub - sujingbo0217/CS222W24-LLM-Attack: Repository for UCR CS222 Natural Language Processing Winter 2024 (1 star)
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets